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INTRODUCTION 

This  is  one  of  a  group  of  handbooks  covering 
the  engineering  information  and  quantitative 
data  needed  in  the  design,  development,  construc¬ 
tion,  and  test  of  military  equipment  which  (as  a 
group)  constitute  the  Army  Materiel  Command 
Engineering  Design  Handbook. 

PURPOSE  OF  HANDBOOK 

The  Handbook  on  Experimental  Statistics  has 
been  prepared  as  an  aid  to  scientists  and  engi¬ 
neers  engaged  in  Army  research  and  develop¬ 
ment  programs,  and  especially  as  a  guide  and 
ready  reference  for  military  and  civilian  person¬ 
nel  who  have  responsibility  for  the  planning  and 
interpretation  of  experiments  and  tests  relating 
to  the  performance  of  Army  equipment  in  the 
design  and  developmental  stages  of  production. 

SCOPE  AND  USE  OF  HANDBOOK 

This  Handbook  is  a  collection  of  statistical 
procedures  and  tables.  It  is  presented  in  five 
sections,  viz : 

AMCP  706-110,  Section  1,  Basic  Concepts 
and  Analysis  of  Measurement  Data  (Chapters 
1-6) 

AMCP  706-111,  Section  2,  Analysis  of  Enu- 
merative  and  Classifies  tory  Data  (Chapters 
7-10) 

AMCP  706-112,  Section  3,  Planning  and 
Analysis  of  Comparative  Experiment  (Chapters 
11-14) 

AMCP  706-113,  Section  4,  Special  Topics 
(Chapters  15-23) 

AMCP  706-114,  Section  5,  Tables 

Section  1  provides  an  elementary  introduc¬ 
tion  to  basic  statistical  concepts  and  furnishes 
full  details  on  standard  statistical  techniques 
for  the  analysis  and  interpretation  of  measure¬ 


ment  data.  Section  2  provides  detailed  pro¬ 
cedures  for  the  analysis  and  interpretation  of 
enumerative  and  classifieatory  data.  Section  3 
has  to  do  with  the  planning  and  analysis  of  com¬ 
parative  experiments.  Section  4  is  devoted  to 
consideration  and  exemplification  of  a  number 
of  important  but  as  yet  non-standard  statistical 
techniques,  and  to  discussion  of  various  other 
special  topics.  An  index  for  the  material  in  all 
four  sections  is  placed  at  the  end  of  Section  4. 
Section  5  contains  all  the  mathematical  tables 
needed  for  application  of  the  procedures  given 
in  Sections  1  through  4. 

An  understanding  of  a  few  basic  statistical 
concepts,  as  given  in  Chapter  1,  is  necesssary; 
otherwise  each  of  the  first  four  sections  is  largely 
independent'  of  the  others.  Each  procedure,  test, 
and  technique  described  is  illustrated  by  means 
of  a  worked  example.  A  list  of  authoritative 
references  is  included,  where  appropriate,  at  the 
end  of  each  chapter.  Step-by-step  instructions 
are  given  for  attaining  a  stated  goal,  and  the 
conditions  under  which  a  particular  procedure  is 
strictly  valid  are  stated  explicitly.  An  attempt  is 
made  to  indicate  the  extent  to  which  results  ob¬ 
tained  by  a  given  procedure  are  valid  to  a  good 
approximation  when  these  conditions  are  not 
fully  met.  Alternative  procedures  are  given  for 
handling  eases  where  the  more  standard  proce¬ 
dures  cannot  be  trusted  to  yield  reliable  results. 

The  Handbook  is  intended  for  the  user  with 
an  engineering  background  who,  although  he  has 
an  occasional  need  for  statistical  techniques,  does 
not  have  the  time  or  inclination  to  become  an  ex¬ 
pert  on  statistical  theory  and  methodology. 

The  Handbook  has  been  written  with  three 
types  of  users  in  mind.  The  first  is  the  person 
who  has  had  a  course  or  two  in  statistics,  and 
who  may  even  have  had  some  practical  experi¬ 
ence  in  applying  statistical  methods  in  the  past, 
but  who  does  not  have  statistical  ideas  and  tech¬ 
niques  at  his  fingertips.  For  him,  the  Handbook 
will  provide  a  ready  reference  source  of  once 
familiar  ideas  and  techniques.  The  second  is  the 
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person  who  feeis,  or  has  been  advised,  that  some 
particular  problem  can  be  solved  by  means  of 
fairly  simple  statistical  techniques,  and  is  in  need 
of  a  book  that  will  enable  him  to  obtain  the  so¬ 
lution  to  his  problem  with  a  minimum  of  outside 
assistance.  The  Handbook  should  enable  such  a 
person  to  become  familiar  with  the  statistical 
ideas,  and  reasonably  adept  at  the  techniques, 
that  are  most  fruitful  in  his  particular  line  of  re¬ 
search  and  development  work.  Finally,  there  is 
the  individual  who,  as  the  head  of,  or  as  a  mem¬ 
ber  of  a  service  group,  has  responsibility  for  ana¬ 
lyzing  and  interpreting  experimental  and  test 
data  brought  in  by  scientists  and  engineers  en¬ 
gaged  in  Army  research  and  development  work. 
This  individual  needs  a  ready  source  of  model 
work  sheets  and  worked  examples  corresponding 
to  the  more  common  applications  of  statistics,  to 
free  him  from  the  need  of  translating  textbook 
discussions  into  step-by-step  procedures  that  can 
be  followed  by  individuals  having  little  or  no 
previous  experience  with  statistical  methods. 

It  is  with  this  last  need  in  mind  that  some 
of  the  procedures  included  in  the  Handbook  have 
been  explained  and  illustrated  in  detail  twice: 
once  for  the  case  where  the  important  question 
is  whether  the  performance  of  a  new  material, 
product,  or  process  exceeds  an  established  stan¬ 
dard;  and  again  for  the  case  where  the  important 
question  is  whether  its  performance  is  not  up  to 
the  specified  standards.  Small  but.  serious  errors 
are  often  made  in  changing  “greater  than”  pro¬ 
cedures  into  “less  than”  procedures. 
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CHAPTER  1 

SOME  BASIC  STATISTICAL  CONCEPTS  AND 
PRELIMINARY  CONSIDERATIONS 


1-1  INTRODUCTION 


Statistics  deals  with  the  collection,  anal¬ 
ysis,  interpretation,  and  presentation  of 
numerical  data.  Statistical  methods  may  be 
divided  into  two  classes — descriptive  and  in¬ 
ductive.  Descriptive  statistical  methods  are 
those  which  are  used  to  summarize  or  de¬ 
scribe  data.  They  are  the  kind  we  see  used 
everyday  in  the  newspapers  and  magazines. 


Inductive  statistical  methods  are  used  when 
we  wish  to  generalize  from  a  small  body  of 
data  to  a  larger  system  of  similar  data.  The 
generalizations  usually  are  in  the  form  of 
estimates  or  predictions.  In  this  handbook 
we  are  mainly  concerned  with  inductive  sta¬ 
tistical  methods. 


1-2  POPULATIONS,  SAMPLES,  AND  DISTRIBUTIONS 


The  concepts  of  a.  population  and  a  sample 
are  basic  to  inductive  statistical  methods. 
Equally  important  is  the  concept  of  a  distri¬ 
bution. 

Any  finite  or  infinite  collection  of  individ¬ 
ual  things — objects  or  events — constitutes  a 
population.  A  population  (also  known  as  a 
universe)  is  thought  of  not  as  just  a  heap  of 
things  specified  by  enumerating  them  one 
after  another,  but  rather  as  an  aggregate 
determined  by  some  property  that  distin¬ 
guishes  between  things  that  do  and  things 
that  do  not  belong.  Thus,  the  term  popula¬ 
tion  carries  with  it  the  connotation  of  com¬ 
pleteness.  In  contrast,  a  sample,  defined  as 
a  portion  of  a  population,  has  the  connota¬ 
tion  of  incompleteness. 

Examples  of  populations  are : 

(a)  The  corporals  in  the  Marines  on  July 
1,  1966. 


(b)  A  production  lot  of  fuzes. 

(c)  The  rounds  of  ammunition  produced 
by  a  particular  production  process. 

(d)  Fridays  the  13th. 

(e)  Repeated  weighings  of  the  powder 
charge  of  a  particular  round  of  ammunition. 

(f)  Firings  of  rounds  from  a  given  pro¬ 
duction  lot. 

In  examples  (a),  (b),  and  (c),  the  "indi¬ 
viduals"  comprising  the  population  are  ma¬ 
terial  objects  (corporals,  fuzes,  rounds) ;  in 
(d)  they  are  periods  of  time  of  a  very  re¬ 
stricted  type;  and  in  (e)  and  (f)  they  are 
physical  operations.  Populations  (a)  and 
(b)  are  clearly  finite,  and  their  constituents 
are  determined  by  the  official  records  of  the 
Marine  Corps  and  the  appropriate  produc¬ 
tion  records,  respectively.  Populations  (c), 
(d),  and  (e)  are  conceptually  infinite.  Off¬ 
hand,  the  population  example  (f)  would 
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seem  to  be  finite,  because  firing  is  a  destruc¬ 
tive  operation ;  but  in  order  to  allow  for  vari¬ 
ation  in  quality  among  “firings”  performed 
In  accordance  with  the  same  general  proce¬ 
dure  it  is  sometimes  useful,  by  analogy  with 
repetitive  weighings,  to  regard  an  actual 
firing  as  a  sample  of  size  one  from  a  con¬ 
ceptually  infinite  population  of  “possible” 
firings,  any  one  of  which  might  have  been 
associated  with  the  particular  round  con¬ 
ceived.  In  this  connection,  note  that  in  exam¬ 
ples  (e)  and  (f)  the  populations  involved 
are  not  completely  defined  until  the  weighing 
and  firing  procedures  concerned  have  been 
fully  specified. 

Attention  to  some  characteristic  of  the 
individuals  of  a  population  that  is  not  the 
same  for  every  individual  leads  immediately 
io  recognition  of  the  distribution  of  this 
characteristic  in  the  population.  Thus,  the 
heights  of  the  corporals  in  the  Marines  on  x 
July  1,  1956,  the  burning  times  of  a  produc¬ 
tion  lot  of  fuzes,  and  the  outcomes  of  succes¬ 
sive  weighings  of  a  powder  charge  (“ob¬ 
served  weights”  of  the  charge)  are  examples 
of  distributions.  The  presence  or  absence  of 
an  attribute  is  a  characteristic  of  an  indi¬ 
vidual  in  a  population,  such  as  “tatooed”  or 
“not  tatooed”  for  the  privates  in  the  Marines. 
This  kind  of  characteristic  has  a  particularly 
simple  type  of  distribution  in  the  population. 

Attention  to  one,  two,  three,  or  more 
characteristics  for  each  individual  leads  to 
a  univariate,  bivariate,  trivariate,  or  multi¬ 
variate  distribution  in  the  population.  The 
examples  of  populations  given  previously 
were  examples  of  univariate  distributions. 
Simultaneous  consideration  of  the  muzzle 
velocities  and  weights  of  powder  charges  of 
rounds  of  ammunition  from  a  given  produc¬ 
tion  process  determines  a  bivariate  dis¬ 
tribution  of  these  characteristics  in  the 
population.  Simultaneous  recognition  of  the 
frequencies  of  each  of  a  variety  of  different 
types  of  accidents  on  Friday  the  13th  leads 
to  a  multivariate  distribution.  Tn  connection 


with  these  examples,  note  that,  as  a  general 
principle,  the  distribution  of  a  characteristic 
or  a  group  of  characteristics  in  a  population 
is  not  completely  defined  until  the  method  or 
methods  of  measurement  or  enumeration  in¬ 
volved  are  fully  specified. 

The  distribution  of  some  particular  prop¬ 
erty  of  the  individuals  in  a  population  is  a 
collective  property  of  the  population;  and 
so,  also,  are  the  average  and  other  charac¬ 
teristics  of  the  distribution.  The  methods  of 
inductive  statistics  enable  us  to  learn  about 
such  population  characteristics  from  a  study 
of  samples. 

An  example  will  illustrate  an  important 
class  of  derived  distributions.  Suppose  we 
select  10  rounds  of  ammunition  from  a  given 
lot  and  measure  their  muzzle  velocities  when 
the  rounds  are  fired  in  a  given  test  weapon. 
Let  X  be  the  average  muzzle  velocity  of  the 
10  rounds.  If  the  lot  is  large,  there  will  be 
many  different  sets  of  10  rounds  which  could 
have  been  obtained  from  the  lot.  For  each 
such  sample  of  10  rounds,  there  will  corre¬ 
spond  an  average  muzzle  velocity  Xi.  These 
averages,  from  all  possible  samples  of  10, 
themselves  form  a  distribution  of  sample 
averages.  This  kind  of  distribution  is  called 
the  sampling  distribution  of  X  for  samples  of 
size  10  from  the  population  concerned.  Sim¬ 
ilarly,  we  may  determine  the  range  R  of 
muzzle  velocities  (i.e.,  the  difference  between 
the  largest  and  the  smallest)  for  each  of  all 
possible  samples  of  10  rounds  each.  These 
ranges  R>  (i  =  1,  2,  .  .  .)  collectively  deter¬ 
mine  the  sampling  distribution  of  the  range 
of  muzzle  velocities  in  samples  of  size  10 
from  the  population  concerned.  The  methods 
of  inductive  statistics  are  based  upon  the 
mathematical  properties  of  sampling  distri¬ 
butions  of  sample  statistics  such  as  X  and  R. 

Let  us  summarize:  A  population  in  Sta¬ 
tistics  corresponds  to  what  in  Logic  is  termed 
the  "universe  of  discourse” — it’s  what  we 
are  talking  about.  By  the  methods  of  in¬ 
ductive  statistics  we  can  learn,  from  a  study 
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of  samples,  only  about  population  character¬ 
istics — only  about  collective  properties  of  the 
populations  represented  by  the  individuals 
in  the  samples — not  about  characteristics 
of  specific  individuals  with  unique  idiosyn¬ 


crasies.  The  population  studied  may  be  large 
or  small,  but  there  must  be  a  population ;  and 
it  should  be  well  defined.  The  characteristic 
of  interest  must  be  a  collective  property  of 
the  population. 


1-3  STATISTICAL  INFERENCES  AND  SAMPLING 


1-3.1  STATISTICAL  INFERENCES 

If  we  were  willing  or  able  to  examine  an 
entire  population,  our  task  would  be  merely 
that  of  describing  that  population,  using 
whatever  numbers,,  figures,  or  charts  we 
cared  to  use.  Since  it  is  ordinarily  incon¬ 
venient  or  impossible  to  observe  every  item 
in  the  population,  we  take  a  sample — a  por¬ 
tion  of  the  population.  Our  task  is  now  to 
generalize  from  our  observations  on  this 
portion  (which  usually  is  small)  to  the  popu¬ 
lation.  Such  generalizations  about  charac¬ 
teristics  of  a  population  from  a  study  of  one 
or  more  samples  from  the  population  are 
termed  statistical  inferences. 

Statistical  inferences  take  two  forms: 
estimates  of  the  magnitudes  of  population 
characteristics,  and  tests  of  hypotheses  re¬ 
garding  population  characteristics.  Both  are 
useful  for  determining  which  among  two  or 
more  courses  of  action  to  follow  in  practice 
when  the  “correct”  course  is  determined  by 
some  particular  but  unknown  characteristic 
of  the  population. 

Statistical  inferences  all  involve  reaching 
conclusions  about  population  characteristics 
(or  at  least  acting  hs  if  one  had  reached  such 
conclusions)  from  a  study  of  samples  which 
are  known  or  assumed  to  be  portions  of  the 
population  concerned.  Statistical  inferences 
are  basically  predictions  of  what  would  be 
found  to  be  the  case  if  the  parent  populations 
could  be  and  were  fully  analyzed  with  respect 
to  the  relevant  characteristic  or  character¬ 
istics. 

A  simple  example  will  serve  to  bring  out 
a  number  of  essential  features  of  statistical 


inferences  and  the  methods  of  inductive  sta¬ 
tistics.  Suppose  that  four  cards  have  been 
drawn  from  a  deck  of  cards  and  have  been 
found  to  be  the  Ace  of  Hearts,  the  Five  of 
Diamonds,  the  Three  of  Clubs,  and  the  Jack 
of  Clubs.  The  specific  methods  discussed  in 
the  follo%ving  paragraphs  will  be  illustrated 
from  this  example. 

First  of  all,  from  the  example,  we  can 
clearly  conclude  at  once  that  the  deck  con¬ 
tained  at  least  one  Heart,  at  least  one  Dia¬ 
mond,  and  at  least  two  Clubs.  We  also  can 
conclude  from  the  presence  of  the  Five  and 
the  Three  that  the  deck  is  definitely  not  a 
pinochle  deck.  These  are  perhaps  trivial  in¬ 
ferences,  but  their  validity  is  above  question 
and  does  not  depend  in  any  way  on  the 
modus  operandi  of  drawing  the  four  cards. 

In  order  to  be  able  to  make  inferences  of 
a  more  substantial  character,  we  must  know 
the  nature  of  the  sampling  operation  that 
yielded  the  sample  of  four  cards  actually  ob¬ 
tained.  Suppose,  for  example,  that  the  sam¬ 
pling  procedure  was  as  follows:  The  cards 
were  drawn  in  the  order  listed,  each  card 
being  selected  at  random  from  all  the  cards 
present  in  the  deck  when  the  card  was  drawn. 
This  defines  a  hypothetical  population  of 
drawings.  By  using  an  appropriate  tech¬ 
nique  of  inductive  statistics — essentially,  a 
“catalog”  of  all  possible  samples  of  four, 
showing  for  each  sample  the  conclusion  to 
be  adopted  whenever  that  sample  occurs — 
we  can  make  statistical  inferences  about 
properties  of  this  population  of  drawings. 
The  statistical  inferences  made  will  be  rig¬ 
orous  if,  and  only  if,  the  inductive  technique 
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used  is  appropriate  to  the  sampling  proce¬ 
dure  actually  employed. 

Thus,  by  taking  the  observed  proportion 
of  Clubs  as  an  estimate  of  the  proportion  of 
Clubs  in  the  abstract  population  of  drawings, 
we  may  assert:  the  proportion  of  Clubs  is 
50%.  Since  random  sampling  of  the  type 
assumed  assures  that  the  proportion  of  Clubs 
in  the  population  of  drawings  is  the  same 
as  the  proportion  of  Clubs  in  the  deck,  we 
may  assert  with  equal  validity:  the  propor¬ 
tion  of  Clubs  in  the  deck  is  50%.  If  the  deck 
concerned  actually  was  a  standard  bridge 
deck,  then  in  the  present  instance  our  esti¬ 
mate  is  wrong  in  spite  of  being  the  best 
single  estimate  available. 

We  know  from  experience  that  with  sam¬ 
ples  of  four  we  cannot  expect  to  “hit  the  nail 
on  the  head”  every  time.  If  instead  of  at¬ 
tempting  to  make  a  single-number  estimate 
we  had  chosen  to  refer  to  a  “catalog”  of 
interval  estimates  (see,  for  example,  Table 
A-22*),  we  would  have  concluded  that  the 
proportion  of  Clubs  is  between  14%  and 
86%  inclusive,  with  an  expectation  of  being 
correct  9  times  out  of  10.  If  the  deck  was 
in  fact  a  standard  bridge  deck,  then  our 
conclusi<?n  is  correct  in  this  instance,  but  its 
validity  depends  on  whether  the  sampling 
procedure  employed  in  drawing  the  four 
cards  corresponds  to  the  sampling  procedure 
assumed  in  the  preparation  of  the  “catalog” 
of  answers. 

It  is  important  to  notice,  moreover,  that 
strictly  we  have  a  right  to  make  statistical 
inferences  only  with  respect  to  the  hypo¬ 
thetical  population  of  drawings  defined  by 
the  sampling  operation  concerned.  In  the 
present  instance,  as  we  shall  see,  the  sam¬ 
pling  operation  was  so  chosen  that  the  pa¬ 
rameters  (i.e.,  the  proportions  of  Hearts, 
Clubs,  and  Diamonds)  of  the  hypothetical 
population  of  drawings  coincide  with  the 
corresponding  parameters  of  the  deck. 


•  The  A -Tahiti  referenced  in  this  handbook  are 
contained  in  Section  6,  AMCP  706-114. 


Hence,  in  the  present  case,  inferences  about 
the  parameters  of  the  population,  of  draw¬ 
ings  may  be  interpreted  as  inferences  about 
the  composition  of  the  deck.  This  empha¬ 
sizes  the  importance  of  selecting  and  em¬ 
ploying  a  sampling  procedure  such  that  the 
relevant  parameters  of  the  population  of 
drawings  bear  a  known  relation  to  the  cor¬ 
responding  parameters  of  the  real-life  situ¬ 
ation.  Otherwise,  statistical  inferences  with 
respect  to  the  population  of  drawings  carried 
over  to  the  real-life  population  will  be  lack¬ 
ing  in  rigor,  even  though  by  luck  they  may 
sometimes  be  correct. 

1-3.2  RANDOM  SAMPLING 

In  order  to  make  valid  nontrivial  gener¬ 
alizations  from  samples  about  characteristics 
of  the  populations  from  which  they  came, 
the  samples  must  have  been  obtained  by  a 
sampling  scheme  which  insures  two  condi¬ 
tions  : 

(a)  Relevant  characteristics  of  the  popu¬ 
lations  sampled  must  bear  a  known  relation 
to  the  corresponding  characteristics  of  the 
population  of  all  possible  samples  associated 
with  the  sampling  scheme. 

(b)  Generalizations  may  be  drawn  from 
such  samples  in  accordance  with  a  given 
“book  of  rules”  whose  validity  rests  on  the 
mathematical  theory  of  probability. 

If  a  sampling  scheme  is  to  meet  these  two 
requirements,  it  is  necessary  that  the  selec¬ 
tion  of  the  individuals  to  be  included  in  a 
sample  involve  some  type  of  random  selec¬ 
tion,  that  is,  each  possible  sample  must  have 
a  fixed  and  determinate  probability  of  selec¬ 
tion,  (For  a  very  readable  expository  dis¬ 
cussion  of  the  general  principles  of  sampling, 
with  examples  of  some  of  the  more  common 
procedures,  see  the  article  by  Cochran,  Mos- 
teller,  and  Tukeyn).  For  fuller  details  see, 
for  example,  Cochran's  book121. 

The  most  widely  useful  type  of  random 
selection  is  simple  (or  unrestricted)  random 
sampling.  This  type  of  sampling  is  defined 
by  the  requirement  that  each  individual  in 
the  population  has  an  equal  chance  of  being 
the  first  member  of  the  sample;  after  the 
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first  member  is  selected,  each  of  the  remain¬ 
ing  individuals  in  the  population  has  an 
equal  chance  of  being  the  second  member 
of  the  sample ;  and  so  forth.  For  a  sampling 
scheme  to  qualify  as  simple  random  sam¬ 
pling,  it  is  not  sufficient  that  “each  individual 
in  the  population  have  an  equal  chance  of 
appearing  in  the  sample,’’  as  is  sometimes 
said,  but  it  is  sufficient  that  “each  possible 
sample  have  an  equal  chance  of  being  se¬ 
lected?’  Throughout  this  handbook,  we  shall 
assume  that  all  samples  are  random  samples 
in  the  sense  of  having  been  obtained  by  sim¬ 
ple  random  sampling. 

It  cannot  be  overemphasized  that  the  ran¬ 
domness  of  a  sample  is  inherent  in  the  sam¬ 
pling  scheme  employed  to  obtain  the  sample 
and  not  an  intrinsic  property  of  the  sample 
itself.  Experience  teaches  that  it  is  not  safe 
to  assume  that  a  sample  selected  haphaz¬ 
ardly,  without  any  conscious  plan,  can  be 
regarded  as  if  it  had  been  obtained  by  simple 
random  sampling.  Nor  does  it  seem  to  be 
possible  to  consciously  draw  a  sample  at 
random.  As  stated  by  Cochran,  Mosteller, 
and  Tukey<1>, 

We  insist  on  some  semblance  of  mechanical  (dice, 
coins,  random  number  tables,  etc.)  randomization 
before  we  treat  a  sample  from  an  existent  popula¬ 
tion  aa  if  it  were  random.  We  realize  that  if  some¬ 
one  just  “grabs  a  handful,”  the  individuals  in  the 
handful  almost  always  resemble  one  another  (on  the 
average)  more  than  do  the  members  of  a  simple 
random  sample.  Even  if  the  “grabs”  are  randomly 
spread  around  so  that  every  individual  has  an  equal 
chance  of  entering  the  sample,  there  are  difficulties. 
Since  the  individuals  of  grab  samples  resemble  one 
another  more  than  do  individuals  of  random  sam¬ 
ples,  it  follows  (by  a  simple  mathematical  argu¬ 
ment)  that  the  means  of  grab  samples  resemble 
one  another  lern  than  the  means  of  random  samples 
of  the  same  size.  From  a  grab  sample,  therefore, 
we  tend  to  underestimate  the  variability  in  the 
population,  although  we  should  have  to  overestimate 
it  in  order  to  obtain  valid  estimates  of  variability 
of  grab  sample  moans  by  substituting  such  an  esti¬ 
mate  into  the  formula  for  the  variability  of  means 
of  simple  random  samples.  Thus,  using  simple  ran¬ 
dom  sample  formulas  for  grab  sample  means  intro¬ 
duces  a  double  bias,  both  parts  of  which  lead  to  an 
unwarranted  appearance  of  higher  stability. 

Instructions  for  formally  drawing  a  sample 
at  random  from  a  particular  population  are 
given  in  Paragraph  1-4. 


Finally,  it  needs  to  be  noticed  that  a  par¬ 
ticular  sample  often  qualifies  as  “a  sample” 
from  any  one  of  several  populations.  For  ex¬ 
ample,  a  sample  of  n  rounds  from  a  single 
carton  is  a  sample  from  that  carton,  from 
the  production  lot  of  which  the  rounds  in 
that  carton  are  a  portion,  and  from  the  pro¬ 
duction  process  concerned.  By  drawing  these 
rounds  from  the  carton  in  accordance  with 
a  simple  random  sampling  scheme,  we  can 
insure  that  they  are  a  (simple)  random  sam¬ 
ple  from  the  carton,  not  from  the  produc¬ 
tion  lot  or  the  production  process.  Only 
if  the  production  process  is  in  a  "state  of 
statistical  control”  may  our  sample  also  be 
considered  to  be  a  simple  random  sample 
from  the  production  lot  and  the  production 
process.  In  a  similar  fashion,  a  sample  of 
repeated  weighings  can  validly  be  consid¬ 
ered  to  be  a  random  sample  from  the  con¬ 
ceptually  infinite  population  of  repeated 
weighings  by  the  same  procedure  only  if 
the  weighing  procedure  is  in  a  state  of  sta¬ 
tistical  control  (see  Chapter  18,  in  Section  4, 
AMCP  706-113). 

It  is  therefore  important  in  practice  to 
know  from  which  of  several  possible  "par¬ 
ent”  populations  a  sample  was  obtained  by 
simple  random  sampling.  This  population  is 
termed  the  sampled  population,  and  may  be 
quite  different  from  the  population  of  inter¬ 
est,  termed  the  target  population,  to  which 
we  would  like  our  conclusions  to  be  applica¬ 
ble.  In  practice,  they  are  rarely  identical, 
though  the  difference  is  often  small.  A  sam¬ 
ple  from  the  target  population  of  rounds  of 
ammunition  produced  by  a  particular  pro¬ 
duction  process  will  actually  be  a  sample 
from  one  or  more  production  lots  (sampled 
population) ,  and  the  difference  between  sam¬ 
pled  and  target  populations  will  be  smaller 
if  the  sampled  population  comprises  a  larger 
number  of  production  lots.  The  further  the 
sampled  population  is  removed  from  the 
target  population,  the  more  the  burden  of 
validity  of  conclusions  is  shifted  from  the 
shoulders  of  the  statistician  to  those  of  the 
subject  matter  expert,  who  must  place 
greater  and  greater  (and  perhaps  unwar¬ 
ranted)  reliance  on  "other  considerations.” 
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1-4  SELECTION  OF  A  RANDOM  SAMPLE 


As  has  been  brought  out  previously,  the 
method  of  choosing  a  sample  is  an  all-im¬ 
portant  factor  in  determining  what  use  can 
be  made  of  it.  In  order  for  the  techniques 
described  in  this  handbook  to  be  valid  as 
bases  for  making  statements  from  samples 
about  populations,  we  must  have  unrestricted 
random  samples  from  these  populations.  In 
practice,  it  is  not  always  easy  to  obtain  a 
random  sample  from  a  given  population. 
Unconscious  selections  and  biases  tend  to 
enter.  For  this  reason,  it  is  advisable  to  use 
a  table  of  random  numbers  as  an  aid  in  se¬ 
lecting  the  sample.  Two  tables  of  random 
numbers  which  are  recommended  are  by 
L.  H.  C.  Tippett*”  and  The  Rand  Corpora¬ 
tion'”.  These  tables  contain  detailed  instruc¬ 
tions  for  their  use.  An  excerpt  from  one  of 
these  tables'”  is  given  in  Table  A-36.  This 
sample  is  included  for  illustration  only;  a 
larger  table  should  be  used  in  any  actual 
problem.  Repeated  use  of  the  same  portion 
of  a  table  of  random  numbers  will  not 
satisfy  the  requirements  of  randomness. 

An  illustration  of  the  method  of  use  of 
tables  of  random  numbers  follows.  Suppose 
the  population  consists  of  87  items,  and  we 
wish  to  select  a  random  sample  of  10.  Assign 
to  each  individual  a  separate  two-digit  num¬ 
ber  between  00  arid  86.  In  a  table  of  ran¬ 
dom  numbers,  pick  an  arbitrary  starting 
place  and  decide  upon  the  direction  of  read¬ 


ing  the  numbers.  Any  direction  may  be 
used,  provided  the  rule  is  fixed  in  advance 
and  is  independent  of  the  numbers  occurring. 
Read  two-digit  numbers  from  the  table,  and 
select  for  the  sample  those  individuals  whose 
numbers  occur  until  10  individuals  have  been 
selected.  For  example,  in  Table  A-36,  start 
with  the  second  page  of  the  Table  (p.  T-83), 
column  20,  line  6,  and  read  down.  Thfe  10 
items  picked  for  the  sample  would  thus  be 
numbers  38,  44,  13,  73,  39,  41,  35,  07,  14, 
and  47. 

The  method  described  is  applicable  for 
obtaining  simple  random  samples  from  any 
sampled  population  consisting  of  a  finite  set 
of  individuals.  In  the  case  of  an  infinite 
sampled  population,  these  procedures  do  not 
apply.  Thus,  we  might  think  of  the  sampled 
population  for  the  target  population  of 
weighings  as  comprising  all  weighings  which 
might  conceptually  have  been  made  during 
the  time  wjiile  weighing  was  done.  We  can¬ 
not  by  mechanical  randomization  draw  a 
random  sample  from  this  population,  and  so 
must  recognize  that  we  have  a  random  sam¬ 
ple  only  by  assumption.  This  assumption 
will  be  warranted  if  previous  data  indicate 
that  the  weighing  procedure  is  in  a  state  of 
statistical  control ;  unwarranted  if  the  con¬ 
trary  is  indicated ;  and  a  leap  in  the  dark  if 
no  previous  data  are  available. 


1-5  SOME  PROPERTIES  OF  DISTRIBUTIONS 


Although  it  is  unusual  to  examine  popula¬ 
tions  in  their  entirety,  the  examination  of  a 
large  sample  or  of  many  small  samples  from 
a  population  can  give  us  much  information 
about  the  general  nature  of  the  population’s 
characteristics. 

One  device  for  revealing  the  general  na¬ 
ture  of  a  population  distribution  is  a  histo¬ 


gram.  Suppose  we  have  a  large  number  of 
observed  items  and  a  numerical  measure¬ 
ment  for  each  item,  such  as,  for  example,  a 
Rockwell  hardness  reading  for  each  of  5,000 
specimens.  We  first  make  a  table  showing 
the  numerical  measurement  and  the  num¬ 
ber  of  times  (i.e.,  frequency)  this  measure¬ 
ment  was  recorded. 
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Rockwell 

Hardness 

Number 

Frequency 

55 

1 

56 

17 

57 

135 

68 

503 

59 

1,110 

60 

1,470 

61 

1,120 

62 

490 

63 

126 

64 

26 

66 

3 

Data  taken.  by  permission,  from  Sampling  Inspection  by 
Variable*  by  A.  H.  Bowker  and  H.  P.  Good*.  Copyright.  1662* 
McGraw-Hill  Book  Company.  Inc. 


From  this  frequency  table  we  can  make  the 
histogram  as  shown  in  Figure  1-1.  The 
height  of  the  rectangle  for  any  hardness 
range  is  determined  by  the  number  of  items 
in  that  hardness  range.  The  rectangle  is 
centered  at  the  tabulated  hardness  value.  If 
we  take  the  sum  of  all  the  rectangular  areas 
to  be  one  square  unit,  then  the  area  of  an 
individual  rectangle  is  equal  to  the  propor¬ 
tion  of  items  in  the  sample  that  have  hard¬ 
ness  values  in  the  corresponding  range. 
When  the  sample  is  large,  as  in  the  present 
instance,  the  histogram  may  be  taken  to 
exemplify  the  general  nature  of  the  corre¬ 
sponding  distribution  in  the  population. 


If  it  were  possible  to  measure  hardness  in 
finer  intervals,  we  would  be  able  to  draw  a 
larger  number  of  rectangles,  smaller  in 
width  than  before.  For  a  sufficiently  large 
sample  and  a  sufficiently  fine  “mesh,"  we 
would  be  justified  in  blending  the  tops  of  the 
rectangles  into  a  continuous  curve,  such  as 
that  shown  in  Figure  1-2,  which  we  could 
expect  to  more  nearly  represent  the  under¬ 
lying  population  distribution. 


Figure  1-1.  Histogram  representing  the  dis¬ 
tribution  of  5,000  Rockwell  hardness 
readings. 

Reproduced  by  permission  from  Sampling  Inspection  by  Vari¬ 
able*  by  A.  K.  Bowker  and  H.  P.  Goode.  Copyright.  1962, 
McGraw-Hill  Book  Company.  Inc. 


Figure  1-2.  Normal  curve  fitted  to  the  dis¬ 
tribution  of  5,000  Rockwell  hardness 
readings. 

Reproduced  by  permission  from  Sampling  Inspection  by  Vari¬ 
ables  by  A.  H.  Bowker  and  H.  P.  Goode.  Copyright,  1962, 
McGraw-Hill  Book  Company,  Ine. 
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If  we  were  to  carry  out  this  sort  of  scheme 
on  a  large  number  of  populations,  we  would 
find  that  many  different  curves  would  arise, 
as  illustrated  in  Figure  1-3.  Possibly,  the 
majority  of  them  would  resemble  the  class 
of  symmetrical  bell-shaped  curves  called 
“normal”  or  “Gaussian”  distributions,  an  ex¬ 
ample  of  which  is  shown  in  the  center  of 
Figure  1-3.  A  normal  distribution  is  uni- 
modal,  i.e.,  has  only  a  single  highest  point 
or  mode,  as  also  are  the  two  asymmetrical 
curves  in  the  lower  left  and  upper  right  of 
Figure  1-3. 

A  “normal”  distribution  is  completely  de¬ 
termined  by  two  parameters:  to,  the  arith¬ 
metic  mean  (or  simply  “the  mean”)  of  the 
distribution,  and  <r,  the  standard  deviation 
(often  termed  the  “population  mean”  and 
“population  standard  deviation”).  The  vari¬ 
ance  of  the  distribution  is  cr2.  Since  a  nor¬ 
mal  curve  is  both  unimodal  and  symmetrical. 


REVERSE  J  SNAP  ED  CURVE  SKEW  CURVE 

EXTREME  POSITIVE  MODERATE  POSITIVE 

SKEWNESS  SKEWNESS 


Figure  1-8.  Frequency  distributions 
of  various  skapes. 

Adapted  with  permission  from  Element*  of  Stati*t\ca>l  Reasoning 
by  A.  E.  Trelonr,  Copyright.  1939, 

John  Wiley  A  Sona.  Inc. 


Figure  1-4.  Three  different  normal 
distributions. 


to  is  also  the  mode  and  the  value  which  di¬ 
vides  the  area  under  the  curve  in  half,  i.e., 
the  median.  It  is  useful  to  remember  that 
tr  is  the  distance  from  to  to  either  of  the  two 
inflection  points  on  the  curve.  (The  inflec¬ 
tion  point  is  the  point  at  which  the  curve 
changes  from  concave  upward  to  concave 
downward.)  This  is  a  special  property  of 
the  normal  distribution.  More  generally,  the 
mean  of  a  distribution  to  is  the  “center  of 
gravity”  of  the  distribution ;  a-  is  the  “radius 
of  gyration”  of  the  distribution  about  m,  in 
the  language  of  mechanics;  and  cr2  is  the 
second  moment  about  m. 

The  parameter  to  is  the  location  param¬ 
eter  of  a  normal  distribution,  while  cr  is  a 
measure  of  its  spread,  scatter,  or  dispersion. 
Thus,  a  change  in  to  merely  slides  the  curve 
right  or  left  without  changing  its  profile, 
while  a  change  in  cr  widens  or  narrows  the 
curve  without  changing  the  location  of  its 
center.  Three  different  normal  curves  are 
shown  in  Figure  1-4.  (All  normal  curves  in 
this  section  are  drawn  so  that  the  area  un¬ 
der  the  curve  is  equal  to  one,  which  is  a 
standard  convention.) 

Figure  1-6  shows  the  percentage  of  ele¬ 
ments  of  the  population  contained  in  various 
intervals  of  a  normal  distribution,  z  is  the 
distance  from  the  population  mean  in  units  of 
the  standard  deviation  and  is  computed  using 
the  formula  z  =  (X~m)/a-,  where  X  repre¬ 
sents  any  value  in  the  population.  Using  z 
to  enter  Table  A-l,  we  find  P,  the  proportion 
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of  elements  in  the  population  which  have 
values  of  z  smaller  than  any  given  z.  Thus, 
as  shown  in  Fig.  1-5,  34.13%  of  the  popula¬ 
tion  will  have  values  of  z  between  0  and  1 
(or  between  0  and  -1)  ;  13.59%  of  the  popu¬ 
lation,  between  1  and  2  (or  between  -1  and 
—2) ;  2.14%  between  2  and  3  (or  between  —2 
and  -3) ;  and  .14%  beyond  3  (or  beyond  —3). 
Figure  1-5  shows  these  percentages  of  the 
population  in  various  intervals  of  z. 

For  example,  suppose  we  know  that  the 
chamber  pressures  of  a  lot  of  ammunition 
may  be  represented  by  a  normal  distribu¬ 
tion,  with  the  average  chamber  pressure  m  = 
50,000  psi  and  standard  deviation  er  —  5,000 

psi.  Then  z  = — and  we  know  (Fig. 

1-5)  that  if  we  fired  the  lot  of  ammunition 
in  the  prescribed  manner  we  would  expect 
50%  of  the  rounds  to  have  a  chamber  pres¬ 
sure  above  50,000  psi,  15.9%  to  have  pres¬ 
sures  above  55,000  psi,  and  2.3%  to  have 
pressures  above  60,000  psi,  etc. 


z  —  X-m 
a 


Figure  1  -5.  Percentage  of  the  population  in 
various  intervals  of  a  normal  distribution. 
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1-6  ESTIMATION  OF  m  and  (7 


In  areas  where  a  lot  of  experimental  work 
hfts  been  done,  it  often  happens  that  we  know 
m  or  cr,  or  both,  fairly  accurately.  However, 
in  the  majority  of  cases  it  will  be  our  task 
to  estimate  them  by  means  of  a  sample.  Sup¬ 
pose  we  have  n  observations,  Xlt  X2,  .  .  Xn 
taken  at  random  from  a  normal  population. 
From  a  sample,  what  are  the  best  estimates 
of  m  and  cr?  Actually,  it  is  usual  to  com¬ 
pute  the  best  unbiased  estimates  of  m  and  cr2, 
and  then  take  the  square  root  of  the  estimate 
of  <r*  as  the  estimate  of  <r.  These  recom¬ 
mended  estimates  of  m  and  cr 2  are :  * 


X  and  s 1  are  the  sample  mean  and  sample 
estimate  of  variance,  respectively,  (s  is  often 
called  “the  sample  standard  deviation,"  but 
this  is  not  strictly  correct  and  we  shall  avoid 
the  expression  and  simply  refer  to  s.)  For 
computational  purposes,  the  following  for¬ 
mula  for  s2  is  more  convenient: 


*  The  Greek  symbol  Z  is  often  used  as  shorthand 
for  “the  sum  of.”  For  example, 

4 

]T  x,  =  x,  +  x,  +  x,  +  x, 

i-i 

i 

T,  (X,  +  Y,)  =  (X,  +  K,)  +  (X,  +  y,)  +  ( X ,  +  Y.) 

V- 1 

» 

Z)x,y,  =  x,y,  +  x,y, +  x.y, 

i 

T  c-c  +  c+  c-3c 

»-l 


Nearly  every  sample  will  contain  differ¬ 
ent  individuals,  and  thus  the  estimates  X 
and  s 2  of  m  and  a-2  will  differ  from  sample 
to  sample.  However,  these  estimates  are 
such  that  “on  the  average”  they  tend  to  be 
equal  to  m  and  cr2,  respectively,  and  in  this 
sense  are  unbiased.  If,  for  example,  we  have 
a  large  number  of  random  samples  of  size 
n,  the  average  of  their  respective  estimates 
of  cr!  will  tend  to  be  near  cr2.  Furthermore, 
the  amount  of  fluctuation  of  the  respective 
s2’s  about  cr 2  (or  of  the  X’s  about  m,  if  we 
are  estimating  m)  will  be  smaller  in  a  cer¬ 
tain  well-defined  sense  than  the  fluctuation 
would  be  for  any  estimates  other  than  the 
recommended  ones.  For  these  reasons,  X 
and  s 2  are  called  the  “best  unbiased"  esti¬ 
mates  of  m  and  a2,  respectively.* 

As  might  be  expected,  the  larger  the  sam¬ 
ple  size  n,  the  more  faith  we  can  put  in  the 
estimates  X  and  s2.  This  is  illustrated  in 
Figures  1-6  and  1-7.  Figure  1-6  shows  the 
distribution  of  X  (sample  mean)  for  samples 
of  various  sizes  from  the  same  normal  dis¬ 
tribution.  The  curve  for  n  =  1  is  the  distri¬ 
bution  for  individuals  in  the  population.  All 
of  the  curves  are  centered  at  m,  the  popula- 


*  On  the  other  hand,  *  is  not  an  unbiased  esti¬ 
mator  of  a.  Thus,  in  samples  of  size  n  from  a  nor¬ 
mal  distribution,  the  situation  is: 


s  is  an  unbiased 

Sample  size,  n  estimator  of: 
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tion  mean,  but  the  scatter  becomes  less  as  n 
gets  larger.  Figure  1-7  shows  the  distribu¬ 
tion  of  s 2  (sample  variance)  for  samples  of 
various  sizes  from  the  same  normal  distri¬ 
bution. 


Figure  1-6.  Sampling  distribution  of  X  for 
random  samples  of  size  n  from  a  normal 
population  with  mean  m. 

Reproduced  by  permission  from  The  Method*  of  Statistic*. 
4th  ed-.  by  L.  H.  C.  Tippett,  Copyright,  1952,  John  Wiley  6 
Sons,  Inc. 


Figure  1-7.  Sampling  distribution  of  sl  for 
sample  size  n  from  a  normal  population 
with  a  ~  1. 


Adapted  with  permlMlon  from  Some  Theory  of  Somplina,  by 
W.  Edwards  Demlnr.  Copyright,  1950,  John  Wiley  A  Soiu,  Inc. 


1-7  CONFIDENCE  INTERVALS 


Inasmuch  as  estimates  of  m  and  cr  vary 
from  sample  to  sample,  interval  estimates 
of  m  and  <r  may  sometimes  be  preferred  to 
“single-value”  estimates.  Provided  we  have 
a  random  sample  from  a  normal  population, 
we  can  make  interval  estimates  of  m  or  cr 
with  a  chosen  degree  of  confidence.  The  level 
of  confidence  is  not  associated  with  a  par¬ 
ticular  interval,  but  is  associated  with  the 
method  of  calculating  the  interval.  The  in¬ 
terval  obtained  from  a  particular  sample 
either  brackets  the  true  parameter  value 
(m  or  cr,  whichever  we  are  estimating)  or 
does  not.  The  confidence  coefficient  y  is  sim¬ 


ply  the  proportion  of  samples  of  size  n  for 
which  intervals  computed  by  the  prescribed 
method  may  be  expected  to  bracket  m  (or 
cr).  Such  intervals  are  known  as  confidence 
intervals,  and  always  are  associated  with  a 
prescribed  confidence  coefficient.  As  we 
would  expect,  larger  samples  tend  to  give 
narrower  confidence  intervals  for  the  same 
level  of  confidence. 

Suppose  we  are  given  the  lot  of  ammuni¬ 
tion  mentioned  earlier  (Par.  1-5)  and  wish 
to  make  a  confidence  interval  estimate  of 
the  average  chamber  pressure  of  the  rounds 
in  the  lot.  The  true  average  is  50,000  psi, 
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although  this  value  is  unknown  to  us.  Let 
us  take  a  random  sample  of  four  rounds  and 
from  this  sample,  using  the  given  procedure, 
calculate  the  upper  and  lower  limits  for  our 
confidence  interval,  Consider  all  the  possi¬ 
ble  samples  of  size  4  that  could  have  been 


taken,  and  the  resulting  confidence  intervals 
computed  from  each.  If  we  compute  50% 
(90%)  confidence  intervals,  then  we  expect 
50%  (90%)  of  the  computed  intervals  to 
cover  the  true  value,  50,000  psi.  See  Fig¬ 
ure  1-8. 
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CASE  A, 50%  CONFIDENCE  INTERVALS 


Figure  1-8.  Computed  confidence  intervals  for  100  samples  of  size  4  drawn  at  random  from 
a  normal  population  with  m  =  50,000  psi,  a  -  5,000  psi.  Case  A  shows  50%  confidence 
intervals;  Case  B  shows  90%  confidence  intervals. 


Adapted  with  permisHion  from  A  STM  Manual  on  Quality  Control  of  Material*,  Copyright,  1951,  American  Society  for  Testing 

Materials. 
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In  Case  A  of  Figure  1-8,  51  of  the  100 
intervals  actually  include  the  true  mean.  For 
50%  confidence  interval  estimates,  we  would 
expect  in  the  long  run  that  50%  of  the  inter¬ 
vals  would  include  the  true  mean.  Fifty-one 
out  of  100  is  a  reasonable  deviation  from  the 
expected  50%.  In  Case  B,  90  out  of  100  of 
the  intervals  contain  the  true  mean.  This  is 
precisely  the  expected  number  for  90%  inter¬ 
vals. 

Note  also  (Fig.  1-8)  that  the  successive 
confidence  intervals  vary  both  in  position 
and  width.  This  is  because  they  were  com¬ 
puted  (see  Par.  2-1.4)  from  the  sample 


statistics  X  and  s,  both  of  which  vary  from 
sample  to  sample.  If,  on  the  other  hand, 
the  standard  deviation  of  the  population 
distribution  c r  were  known,  and  the  con¬ 
fidence  intervals  were  computed  from  the 
successive  X’s  and  er  (procedure  given  in 
Par.  2-1.5),  then  the  resulting  confidence 
intervals  would  all  be  the  same  width,  and 
would  vary  in  position  only. 

Finally,  as  the  sample  size  increases,  con¬ 
fidence  intervals  tend  not  only  to  vary  less 
in  both  position  and  width,  but  also  to 
"pinch  in”  ever  closer  to  the  true  value  of 
the  population  parameter  concerned,  as  illus¬ 
trated  in  Figure  1-9. 
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Figure  1-9.  Computed  50%  confidence  intervals  for  the  population  mean  m  from  100 
samples  of  4,  4 0  samples  of  100,  and  4  samples  of  1000. 

Adapted  with  permission  from  Statistical  Method  from  the  Viewpoint  of  Ovality  Control  by  W.  A.  Shewhart  (edited  by  W.  Edwards 
Demina).  Copyright,  1M».  Graduate  School.  U.S.  Department  of  Agriculture,  Washington,  D.  C. 
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1-8  STATISTICAL  TOLERANCE  LIMITS 


Sometimes  what  is  wanted  is  not  an  esti- 
itlftte  of  the  mean  and  variance  of  the  popu¬ 
lation  distribution  but,  instead,  two  outer 
values  or  limits  which  contain  nearly  all  of 
the  population  values.  For  example,  if  ex¬ 
tremely  low  chamber  pressures  or  extremely 
high  chamber  pressures  might  cause  serious 
problems,  we  may  wish  to  know  approxi¬ 
mate  limits  to  the  range  of  chamber  pres¬ 
sures  in  a  lot  of  ammunition.  More  spe¬ 
cifically,  we  may  wish  to  know  within  what 
limits  99%,  for  example,  of  the  chamber 
pressures  lie.  If  we  knew  the  mean  m  and 
standard  deviation  cr  of  chamber  pressures 
in  the  lot,  and  if  we  knew  the  distribution 
of  chamber  pressures  to  be  normal  (or  very 
nearly  normal),  then  we  could  take  m  —  3<r 
and  m  +  3<r  as  our  limits,  and  conclude  that 


approximately  99.<7%  of  the  chamber  pres¬ 
sures  lie  within  these  limits  (see  Fig.  1-5). 
If  we  do  not  know  m  and  a-,  then  we  may 
endeavor  to  approximate  the  limits  with 
statistical  tolerance  limits  of  the  form 
X  —  Ks  and  X  +  Ks,  based  on  the  sample 
statistics  X  and  s,  with  K  chosen  so  that  we 
may  expect  these  limits  to  include  at  least  P 
percent  of  the  chamber  pressures  in  the  lot, 
at  some  prescribed  level  of  confidence  a. 

Three  sets  of  such  limits  for  P  =  99.7%, 
corresponding  to  sample  sizes  n  =  4,  100, 
and  1,000,  are  shown  by  the  bars  in  Figure 
1-10.  It  should  be  noted  that  for  samples  of 
size  4,  the  bars  are  very  variable  both  in 
location  and  width,  but  that  for  «  =  100  and 
«  =  1,000,  they  are  of  nearly  constant  width 


0  30  40  00  to  100  0  30  40  0  5 

SAMPLE  NUMBER 


Figure  1-10.  Computed  statistical  tolerance  limits  for  99.7%  of  the  population  from  100 
samples  of  size  4,  1*0  samples  of  size  100,  and  U  samples  of  size  1000. 

Adapted  with  permission  from  Stafuticai  Method  from  the  Viewpoint  of  Quality  Control  by  W.  A.  Shewhart  (edited  by  W.  Edwards 
Demin*).  Copyright,  1939,  Graduate  School,  U.S.  Department  of  Agriculture.  Washington,  D.  C. 
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and  position — and  their  end  points  approxi¬ 
mate  very  closely  to  m  —  3 cr  and  m  +  3o\  In 
other  words,  statistical  tolerance  intervals 
tend  to  a  fixed  size  (which  depends  upon  P) 
as  the  sample  size  increases,  whereas  con¬ 
fidence  intervals  shrink  down  towards  zero 
width  with  increasing  sample  size,  as  illus¬ 
trated  in  Figure  1-9. 

The  difference  in  the  meanings  of  the 
terms  confidence  intervals,  statistical  toler¬ 
ance  limits,  and  engineering  tolerance  limits 


should  be  noted.  A  confidence  interval  is  an 
interval  within  which  we  estimate  a  given 
population  parameter  to  lie  (e.g.,  the  popula¬ 
tion  mean  m  with  respect  to  some  character¬ 
istic)  .  Statistical  tolerance  limits  for  a  given 
population  are  limits  within  which  we  ex¬ 
pect  a  stated  proportion  of  the  population 
to  lie  with  respect  to  some  measurable  char¬ 
acteristic.  Engineering  tolerance  limits  are 
specified  outer  limits  of  acceptability  with 
respect  to  some  characteristic  usually  pre¬ 
scribed  by  a  design  engineer. 


1-9  USING  STATISTICS  TO  MAKE  DECISIONS 


1-9.1  APPROACH  TO  A  DECISION  PROBLEM 

Consider  the  following  more-or-less  typical 
practical  situation:  Ten  rounds  of  a  new 
type  of  shell  are  fired  into  a  target,  and  the 
depth  of  penetration  is  measured  for  each 
round.  The  depths  of  penetration  are  10.0, 
11.1,  10.5,  10.5,  11.2,  10.8,  9.8,  12.2,  11.0, 
and  9.9  cm.  The  average  penetration  depth 
of  the  comparable  standard  shell  is  10.0  cm. 
We  wish  to  know  whether  the  new  type  shells 
penetrate  farther  on  the  average  than  the 
standard  type  shells. 

If  we  compute  the  arithmetic  mean  of  the 
ten  shells,  we  find  it  is  10.70  cm.  Our  first 
impulse  might  be  to  state  that  on  the  aver¬ 
age  the  new  shell  will  penetrate  0.7  cm. 
farther  than  the  standard  shell.  This,  in¬ 
deed,  is  our  best  single  guess,  but  how  sure 
can  we  be  that  this  actually  is  close  to  the 
truth?  One  thing  that  might  catch  our  notice 
is  the  variability  in  the  individual  penetra¬ 
tion  depths  of  the  new  shells.  They  range 
from  9.8  cm.  to  12.2  cm.  The  standard  devi¬ 
ation  as  measured  by  s  calculated  from  the 
sample  is  0.73  cm.  Might  not  our  sample  of 
ten  shells  have  contained  some  atypical  ones 
of  the  new  type  which  have  unusually  high 
penetrating  power?  Could  it  be  that  the  new 
shell  is,  on  the  average,  no  "better  than  the 
standard  one?  If  we  were  obliged  to  decide, 


on  th^  basis  of  the  results  obtained  from 
these  ten  shells  alone,  whether  to  keep  on 
making  the  standard  shells  or  to  convert 
our  equipment  to  making  the  new  shell,  how 
can  we  make  a  valid  choice? 

A  very  worthwhile  step  toward  a  solution 
in  such  situations  is  to  compute,  from  the 
data  in  hand,  a  confidence  interval  for  the 
unknown  value  of  the  population  parameter 
of  interest.  The  procedure  (given  in  Par. 
2-1.4)  applied  to  the  foregoing  depth-of- 
penetration  data  for  the  new  type  of  shell 
yields  the  interval  from  10.18  to  11.22  cm. 
as  a  95%  confidence  interval  for  the  popu¬ 
lation  mean  depth  of  penetration  of  shells 
of  the  new  type.  Inasmuch  as  this  interval 
lies  entirely  to  the  right  of  the  mean  for 
the  standard  shell,  10.00  cm.,  we  are  jus¬ 
tified  in  concluding  that  the  new  shell  is, 
on  the  average,  better  than  the  standard, 
with  only  a  5%  risk  of  being  in  error. 
Nevertheless,  taking  other  considerations 
into  account  (e.g.,  cost  of  the  new  type,  cost 
of  changing  over,  etc.),  we  may  conclude 
finally  that  the  improvement — which  may  be 
as  little  as  0.18  cm.,  and  probably  not  more 
than  1.22  cm. — is  not  sufficient  to  warrant 
conversion  to  the  new  type.  On  the  other 
hand,  the  evidence  that  the  new  type  is  al¬ 
most  certainly  better  plus  the  prospect  that 
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the  improvement  may  be  as  great  as  1.22  cm. 
may  serve  to  recommend  further  develop¬ 
mental  activity  in  the  direction  “pioneered” 
by  the  new  type. 

A  somewhat  different  approach,  which 
provides  a  direct  answer  to  our  question 
“Could  it  be  that  the  new  shell  is  on  the 
average  no  better  than  the  standard?”  but 
not  to  the  question  of  whether  to  convert  to 
the  new  type,  is  to  carry  out  a  so-called  test 
of  significance  (or  test  of  a  statistical  hy¬ 
pothesis).  In  the  case  of  the  foregoing  ex¬ 
ample,  the  formal  procedure  for  the  corre¬ 
sponding  test  of  significance  (Par.  3-2.2. 1) 
turns  out  to  be  equivalent  (as  explained  in 
AMCP  706-113,  Chapter  21)  to  noting  whether 
or  not  the  confidence  interval  computed  does 
or  does  not  include  the  population  mean  for 
the  standard  shell  (10.0  cm.).  If,  as  in  the 
present  instance,  the  population  mean  for 
the  standard  shell  is  not  included,  this  is 
taken  to  be  a  negative  answer  to  our  ques¬ 
tion.  In  other  words,  this  is  taken  to  be 
conclusive  evidence  (at  the  5%  level  of  sig¬ 
nificance)  against  the  null  hypothesis  that 
“the  new  shell  is  on  the  average  no  better 
than  the  standard.”  Rejection  of  the  null 
hypothesis  in  this  case  is  equivalent  to  ac¬ 
cepting  the  indefinite  alternative  hypothesis 
that  “the  new  shell  is  better  on  the  average 
than  the  standard.”  If,  on  the  other  hand, 
the  population  mean  for  the  standard  shell 
is  included  in  the  confidence  interval,  this 
is  taken  as  an  affirmative  answer  to  our 
question — not  in  the  positive  sense  of  defi¬ 
nitely  confirming  the  null  hypothesis  (“is 
no  better”),  but  in  the  more-or-less  neutral 
sense  of  the  absence  of  conclusive  evidence 
to  the  contrary. 

As  the  foregoing  example  illustrates,  an 
advantage  of  the  confidence-interval  ap¬ 
proach  to  a  decision  problem  is  that  the  con¬ 
fidence  interval  gives  an  indication  of  how 
large  the  difference,  if  any,  is  likely  to  be, 
and  thus  provides  some  of  the  additional 
information  usually  needed  to  reach  a  final 
decision  on  the  action  to  be  taken  next.  For 
many  purposes,  this  is  a  real  advantage  of 
confidence  intervals  over  tests  of  significance. 


However,  all  statistical  decision  problems 
are  not  amenable  to  solution  via  confidence 
intervals.  For  instance,  the  question  at  issue 
may  be  whether  or  not  two  particular  char¬ 
acteristics  of  shell  performance  are  mutually 
independent.  In  such  a  situation,  any  one 
of  a  variety  of  tests  of  significance  can  be 
used  to  test  the  null  hypothesis  of  “no  de¬ 
pendence.”  Some  of  these  may  have  a  rea¬ 
sonably  good  chance  of  rejecting  the  null 
hypothesis,  and  thus  “discovering”  the  ex¬ 
istence  of  a  dependence  when  a  dependence 
really  exists — even  though  the  exact  nature 
of  the  dependence,  if  any,  is  not  understood 
and  a  definitive  measure  of  the  extent  of 
the  dependence  in  the  population  is  lacking. 

A  precise  test  of  significance  will  be  possi¬ 
ble  if :  (a)  the  sampling  distribution  of  some 
sample  statistic  is  known  (at  least  to  a  good 
approximation)  for  the  case  of  “no  depend¬ 
ence”;  and  (b)  the  effect  of  dependence  on 
this  statistic  is  known  (e.g.,  tends  to  make 
it  larger).  For  a  confidence-interval  ap¬ 
proach  to  be  possible,  two  conditions  are 
necessary:  (a)  there  must  be  agreement  on 
what  constitutes  the  proper  measure  (pa¬ 
rameter)  of  dependence  of  the  two  charac¬ 
teristics  in  the  population;  and,  (b)  there 
must  be  a  sample  estimate  of  this  depend¬ 
ence  parameter  whose  sampling  distribution 
is  known,  to  a  good  approximation  at  least, 
for  all  values  of  the  parameter.  Confidence 
intervals  tend  to  provide  a  more  complete 
answer  to  statistical  decision  problems  when 
they  are  available,  but  tests  of  significance 
are  of  wider  applicability. 

1-9.2  CHOICE  OF  NULL  AND  ALTERNATIVE 
HYPOTHESES 

A  statistical  test  always  involves  a  null 
hypothesis,  which  is  considered  to  be  the 
hypothesis  under  test,  as  against  a  class  of 
alternative  hypotheses.  The  null  hypothesis 
acts  as  a  kind  of  “origin”  or  “base”  (in  the 
sense  of  "base  line”),  from  which  the  alter¬ 
native  hypotheses  deviate  in  one  way  or  an¬ 
other  to  greater  and  lesser  degrees.  Thus, 
in  the  case  of  the  classical  problem  of  the 
tossing  of  a  coin,  the  null  or  base  hypothesis 
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specifies  that  the  probability  of  “heads”  on 
any  single  trial  equals  1/2.  If,  in  a  par¬ 
ticular  situation,  the  occurrence  of  “heads" 
were  an  advantage,  then  we  might  be  par¬ 
ticularly  interested  in  the  one-sided  class  of 
alternative  hypotheses  that  the  probability 
of  “heads"  on  any  single  trial  equals  P, 
where  P  is  some  (unknown)  fraction  ex¬ 
ceeding  1/2.  If  neither  “heads”  nor  ‘tails” 
were  intrinsically  advantageous,  but  a  bias 
in  favor  of  either  could  be  employed  to  ad¬ 
vantage,  then  we  could  probably  be  inter¬ 
ested  in  the  more  general  two-sided  class 
of  alternative  hypotheses  specifying  that  the 
probability  of  “heads”  on  any  single  toss 
equals  P,  where  P  is  some  fraction  (less 
than,  or  greater  than,  but)  not  equal  to  1/2. 

The  important  point  is  that  the  null  hy¬ 
pothesis  serves  as  an  origin  or  base.  In  the 
coin-tossing  instance,  it  also  happens  to  be 
a  favored,  or  traditional,  hypothesis.  This 
is  merely  a  characteristic  of  the  example 
selected.  Indeed,  the  null  hypothesis  is  often 
the  very  antithesis  of  what  we  would  really 
like  to  be  the  case. 

1-9.3  TWO  KINDS  OF  ERRORS 

In  basing  decisions  on  the  outcomes  of 
statistical  tests,  we  always  run  the  risks  of 
making  either  one  or  the  other  of  two  types 
of  error.  If  we  reject  the  null  hypothesis 
when  it  is  true,  e.g.,  announce  a  difference 
which  really  does  not  exist,  then  we  make  an 
Error  of  the  First  Kind.  If  we  fail  to  reject 
a  null  hypothesis  when  it  is  false,  e.g.,  fail 
to  find  an  improvement  in  the  new  shell  over 
the  old  when  an  improvement  exists,  then 
we  make  what  is  called  an  Error  of  the  Sec¬ 
ond  Kind.  Although  we  do  not  know  in  a 
given  instance  whether  we  have  made  an 
error  of  either  kind,  we  can  know  the  prob¬ 
ability  of  making  either  type  of  error. 

1-9.4  SIGNIFICANCE  LEVEL  AND  OPERATING 
CHARACTERISTIC  (OC)  CURVE  OF 
A  STATISTICAL  TEST 

The  risk  of  making  an  error  of  the  first 
kind,  a,  equals  what  is  by  tradition  called 


the  level  of  significance  of  the  test.  The  risk 
of  making  an  error  of  the  second  kind,  /3, 
varies,  as  one  would  expect,  with  the  magni¬ 
tude  of  the  real  difference,  and  is  summa¬ 
rized  by  the  Operating  Characteristic  (OC) 
Curve  of  the  test.  See,  for  example,  Figure 
3-5.  Also,  the  risk  (i  of  making  an  error  of 
the  second  kind  increases  as  the  risk  a  of 
making  an  error  of  the  first  kind  decreases. 
Compare  Figure  3-5  with  Figure  3-6.  Only 
with  “large”  samples  can  we  “have  our  cake 
and  eat  it  too” — and  then  there  is  the  cost 
of  the  test  to  worry  about. 

1-9.5  CHOICE  OF  THE  SIGNIFICANCE  LEVEL 

The  significance  level  of  a  statistical  test 
is  essentially  an  expression  of  our  reluctance 
to  give  hp  or  “reject”  the  null  hypothesis. 
If  we  adopt  a  "stiff”  significance  level,  0.01 
or  even  0.001,  say,  this  implies  that  we  are 
very  unwilling  to  reject  the  null  hypothesis 
unjustly.  A  consequence  of  our  ultracon¬ 
servatism  in  this  respect  will  usually  be  that 
the  probability  of  not  rejecting  the  null  hy¬ 
pothesis  when  it  is  really  false  will  be  large 
unless  the  actual  deviation  from  the  null 
hypothesis  is  large.  This  is  clearly  an  en¬ 
tirely  satisfactory  state  of  affairs  if  we  are 
quite  satisfied  with  the  status  quo  and  are 
only  interested  in  making  a  change  if  the 
change  represents  a  very  substantial  im¬ 
provement.  For  example,  we  may  be  quite 
satisfied  with  the  performance  of  the  stand¬ 
ard  type  of  shell  in  all  respects,  and  not  be 
willing  to  consider  changing  to  the  new  type 
unless  the  mean  depth  of  penetration  of  the 
new  type  were  at  least,  say,  20%  better 
(12.0  cm.). 

On  the  other  hand,  the  standard  shell  may 
be  unsatisfactory  in  a  number  of  respects 
and  the  question  at  issue  may  be  whether 
the  new  type  shows  promise  of  being  able 
to  replace  it,  either  “as  is”  or  with  further 
development.  Here  “rejection”  of  the  null 
hypothesis  would  not  imply  necessary  aban¬ 
donment  of  the  standard  type  and  shifting 
over  to  the  new  type,  but  merely  that  the 
new  type  shows  "promise"  and  warrants 
further  investigation.  In  such  a  situation, 
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one  could  afford  a  somewhat  higher  risk  of 
rejecting  the  null  hypothesis  falsely,  and 
would  take  a  =  0.05  or  0.10  (or  even  0.20, 
perhaps),  in  the  interest  of  increasing  the 
chances  of  detecting  a  small  but  promising 
improvement  with  a  small-scale  experiment. 
In  such  exploratory  work,  it  is  often  more 
important  to  have  a  good  chance  of  detecting 
a  small  but  promising  improvement  than  to 
protect  oneself  against  crying  “wolf,  wolf” 
occasionally — because  the  “wolf,  wolf”  will 
be  found  out  in  due  course,  but  a  promising 
approach  to  improvement  could  be  lost  for¬ 
ever. 

In  summary,  the  significance  level  a  of 
a  statistical  test  should  be  chosen  in  the 
light  of  the  attending  circumstances,  includ¬ 
ing  costs.  We  are  sometimes  limited  in  the 
choice  of  significance  level  by  the  availability 
of  necessary  tables  for  some  statistical  tests. 
Two  values  of  a,  a  =  .05  and  a  =  .01,  have 
been  most  frequently  used  in  research  and 
development  work;  and  are  given  in  tabula¬ 
tions  of  test  statistics.  We  have  adopted 
these  “standard”  levels  of  significance  for 
the  purposes  of  this  handbook. 

1-9.6  A  WORD  OF  CAUTION 

Many  persons  who  regularly  employ  sta¬ 
tistical  tests  in  the  interpretation  of  research 
and  development  data  do  not  seem  to  realize 
that  all  probabilities  associated  with  such 
tests  are  calculated  on  the  supposition  that 
some  definite  set  of  conditions  prevails. 
Thus,  a,  the  level  of  significance  (or  proba¬ 
bility  of  an  error  of  the  first  kind),  is  com¬ 
puted  on  the  assumption  that  the  null  hy¬ 
pothesis  is  strictly  true  in  all  respects;  and 
/3,  the  risk  of  an  error  of  the  second  kind, 
is  computed  on  the  assumption  that  a  par¬ 
ticular  specific  alternative  to  the  null  hypoth¬ 
esis  is  true  and  that  the  statistical  test  con¬ 
cerned  is  carried  out  at  the  a-level  of  signifi¬ 
cance.  Consequently,  whatever  may  be  the 
actual  outcome  of  a  statistical  test,  it  is 
mathematically  impossible  to  infer  from  the 


outcome  anything  whatsoever  about  the  odds 
for  or  against  some  particular  set  of  condi¬ 
tions  being  the  truth. 

Indeed,  it  is  astonishing  how  often  errone¬ 
ous  statements  of  the  type  “since  r  exceeds 
the  1  %  level  of  significance,  the  odds  are  99 
to  1  that  there  is  a  correlation  between  the 
variables”  occur  in  research  literature.  How 
ridiculous  this  type  of  reasoning  can  be  is 
brought  out  by  the  following  simple  exam¬ 
ple <s>:  The  American  Experience  Mortality 
Table  gives  .01008  as  the  probability  of  an 
individual  aged  41  dying  within  the  year. 
If  we  accept  this  table  as  being  applicable  to 
living  persons  today  (which  is  analogous  to 
accepting  the  published  tables  of  the  signif¬ 
icance  levels  of  tests  which  we  apply  to  our 
data),  and  if  a  man’s  age  really  is  41,  then 
the  odds  are  99  to  1  that  he  will  live  out  the 
year.  On  the  other  hand,  if  we  accept  the 
table  and  happen  to  hear  that  some  promi¬ 
nent  individual  has  just  died,  then  we  cannot 
(and  would  not )  conclude  that  the  odds  are 
99  to  1  that  his  age  was  different  from  41. 

Suppose,  on  the  other  hand,  that  in  some 
official  capacity  it  is  our  practice  to  check 
the  accuracy  of  age  statements  of  all  persons 
who  say  they  are  41  and  then  die  within  the 
year.  This  practice  (assuming  the  applica¬ 
bility  of  the  American  Experience  Mortality 
Table)  will  lead  us  in  the  long  run  to  suspect 
unjustly  the  word  of  one  person  in  100  whose 
age  was  41,  who  told  us  so,  and  who  then 
was  unfortunate  enough  to  die  within  the 
year.  The  level  of  significance  of  the  test  is 
in  fact  0.01008  (1  in  100).  On  the  other 
hand,  this  practice  will  also  lead  us  to  dis¬ 
cover  mis-statements  of  age  of  all  persons 
professing  to  be  41  wrho  are  really  some  other 
age  and  who  happen  to  die  within  the  year. 
The  probabilities  of  our  discovering  such 
mis-statements  will  depend  on  the  actual 
ages  of  the  persons  making  them.  We  shall, 
however,  let  slip  by  as  correct  all  statements 
“age  41”  corresponding  to  individuals  who 
are  not  41  but  who  do  not  happen  to  die 
within  the  year. 
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The  moral  of  this  is  that  all  statistical 
tests  can  and  should  be  viewed  in  terms  of 
the  consequences  which  may  be  expected  to 
ensue  from  their  repeated  use  in  suitable 
circumstances.  When  viewed  in  this  light, 


the  great  risks  involved  in  drawing  conclu¬ 
sions  from  exceedingly  small  samples  be¬ 
comes  manifest  to  anyone  who  takes  the 
time  to  study  the  OC  curves  for  the  statis¬ 
tical  tests  in  common  use. 
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DISCUSSION  OF  TECHNIQUES 
IN  CHAPTERS  2  THROUGH  6 


The  techniques  described  in  Chapters  2 
through  6  apply  to  the  analysis  of  results  of 
experiments  expressed  as  measurements  in 
some  conventional  units  on  a  continuous 
scale.  They  do  not  apply  to  the  analysis  of 
data  in  the  form  of  proportions,  percentages, 
or  counts. 

It  is  assumed  that  the  underlying  popula¬ 
tion  distributions  are  normal  or  nearly 
normal.  Where  this  assumption  is  not  very  im¬ 
portant,  or  where  the  actual  population  dis¬ 
tribution  would  show  only  slight  departure 
from  normality,  an  indication  is  given  of  the 
effect  upon  the  conclusions  derived  from  the 
use  of  the  techniques.  Where  the  normality 
assumption  is  critical,  or  where  the  actual 
population  distribution  shows  substantial  de¬ 
parture  from  normality,  or  both,  suitable 
warnings  are  given. 


Table  A-37  is  a  table  of  three-decimal-place 
random  normal  deviates  that  exemplify  sam¬ 
pling  from  a  normal  distribution  with  zero 
mean  (m  =  0)  and  unit  standard  deviation 
(o-  =  1) .  To  construct  numbers  that  will  sim¬ 
ulate  measurements  that  are  normally  dis¬ 
tributed  about  a  true  value  of,  say,  0.12,  with 
a  standard  deviation  of,  say,  0.02,  multiply 
the  table  entries  by  0.02  and  then  add  0.12. 
The  reader  who  wishes  to  get  a  feel  for  the 
statistical  behavior  of  sample  data,  and  to 
try  out  and  judge  the  usefulness  of  particu¬ 
lar  statistical  techniques,  is  urged  to  carry 
out  a  few  “dry  runs”  with  such  simulated 
measurements  of  known  characteristics. 

All  A-Tables  referenced  in  these  Chapters 
are  contained  in  AMCP  706-114,  Section  6, 
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CHAPTER  2 

CHARACTERIZING  THE  MEASURED  PERFORMANCE  OF 
A  MATERIAL,  PRODUCT,  OR  PROCESS 


2-1  ESTIMATING  AVERAGE  PERFORMANCE  FROM  A  SAMPLE 


2-1.1  GENERAL 

In  this  Chapter  we  present  two  important 
kinds  of  estimates  of  the  average  perform¬ 
ance  of  a  material,  product,  or  process  from 
a  sample.  These  include  the  best  single  esti¬ 
mate,  and  confidence  interval  estimates.* 

Specific  procedures  are  given  for  obtaining 
confidence  interval  estimates  when : 

(a)  we  have  a  sample  from  a  normal 
population  whose  variability  is  unknown; 
and, 

(b)  we  have  a  sample  from  a  normal 
population  whose  variability  is  known. 

When  the  departures  from  normality  are 
not  great,  or  when  the  sample  sizes  are 
moderately  large,  interval  estimates  made 
as  described  in  Paragraphs  2-1.4  and  2-1.6 
will  have  confidence  levels  very  little  differ¬ 
ent  from  the  chosen  or  nominal  level. 

The  following  data  will  serve  to  illustrate 
the  application  of  the  procedures. 

Data  Sample  2-1 — Thickness  of  Mica  Waihort 

Form:  Measurements  X„  Xt,  X„  of  n 
items  selected  independently  at  ran¬ 
dom  from  a  much  larger  group. 


*  The  reader  who  is  not  familiar  with  the  mean¬ 
ing  and  interpretation  of  confidence  intervals  should 
refer  to  Chapter  1,  and  to  Paragraph  2-1.8  of  this 
Chapter. 


Example:  Ten  mica  washers  are  taken  at 
random  from  a  large  group,  and  their 
thicknesses  measured  in  inches: 


.123 

.132 

.124 

.123 

.126 

.126 

.129 

.129 

.120 

.128 

In  general,  what  can  we  say  about  the 
larger  group  on  the  basis  of  our  sample? 
We  show  how  to  answer  two  questions : 

(a)  What  is  our  best  guess  as  to  the  aver¬ 
age  thickness  in  the  whole  lot? 

(b)  Can  we  give  an  interval  which  we 
expect,  with  certain  confidence,  to  bracket 
the  true  average — i.e.,  a  confidence  interval ? 

These  two  questions  are  answered  in  the 
paragraphs  which  follow,  using  the  data 
shown  above.  Another  question,  which  is 
sometimes  confused  with  (b)  above,  is 
treated  in  Paragraph  2-5.  This  is  the  ques¬ 
tion  of  setting  statistical  tolerance  limits,  or 
estimating  an  interval  which  will  include, 
with  prescribed  confidence,  a  specified  pro¬ 
portion  of  the  individual  items  in  the  popu¬ 
lation. 

2-1.2  BEST  SINGLE  ESTIMATE 

The  most  common  and  ordinarily  the  best 
single  estimate  of  the  population  mean  m  is 
simply  the  arithmetic  mean  of  the  measure¬ 
ments. 
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Procedure 

Compute  the  arithmetic  mean  X  of  the  n 
measurements  Xir  X3, . . . ,  Xn. 


Example 

Compute  the  arithmetic  mean  X  of  10 
measurements  (Data  Sample  2-1): 

v  _  .123  +  .124  +  .126  +  •  ■  •  4-  .128 
A  _  10 
_  1.260 
10 

=  .1260  inch 

2-1.3  SOME  REMARKS  ON  CONFIDENCE 
INTERVAL  ESTIMATES 

When  we  take  a  sample  from  a  lot  or  a 
population,  the  sample  average  will  seldom 
be  exactly  the  same  as  the  lot  or  population 
average.  We  do  hope  that  it  is  fairly  close, 
and  we  would  like  to  state  an  interval  which 
we  are  confident  will  bracket  the  lot  mean. 
If  we  made  such  interval  estimates  in  a 
particular  fashion  a  large  number  of  times, 
and  found  that  these  intervals  actually  did 
contain  the  true  mean  in  99%  of  the  cases, 
we  might  say  that  we  were  operating  at  a 
99%  confidence  level.  Our  particular  kind 


of  interval  estimates  might  likewise  be  called 
“99%  confidence  intervals.”  Similarly,  if 
our  intervals  included  the  true  average 
“95%  of  the  time” — strictly,  in  95%  of  the 
times  or  instances  involved — we  would  be 
operating  at  a  95%  confidence  level,  and  our 
intervals  would  be  called  95%  confidence  in¬ 
tervals.  In  general,  if  in  the  long  run  we 
expect  100(1  -  a)  %  of  our  intervals  to  con¬ 
tain  the  true  value,  we  are  operating  at  the 
100(1  —  a)  %  confidence  level. 

We  may  choose  whatever  confidence  level 
we  wish.  Confidence  levels  y  commonly  used 
are  99%  and  95%,  which  correspond  to 
a  =  .01  and  a  =  .05.  If  we  wish  to  estimate 
the  mean  of  some  characteristic  of  a  large 
group  (population)  using  the  results  of  a 
random  sample  from  that  group,  the  proce¬ 
dures  of  Paragraphs  2-1.4  and  2-1.5  will 
allow  us  to  make  interval  estimates  at  any 
chosen  confidence  level.  It  is  assumed  that 
the  characteristic  of  interest  has  a  normal 
distribution  in  the  population.  We  may  elect 
to  make  a  two-sided  interval  estimate,  ex¬ 
pected  to  bracket  the  mean  from  both  above 
and  below ;  or  we  may  make  a  one-sided  in¬ 
terval  estimate,  limited  on  either  the  upper 
or  the  lower  side,  which  is  expected  to  con¬ 
tain  the  mean  and  to  furnish  either  an  upper 
or  a  lower  bound  to  its  magnitude. 


2-1.4  CONFIDENCE  INTERVALS  FOR  THE  POPULATION  MEAN  WHEN  KNOWLEDGE 
OF  THE  VARIABILITY  CANNOT  BE  ASSUMED 

2-1. 4.1  Two-Sided  Confidence  Interval 

This  procedure  gives  an  interval  which  is  expected  to  bracket  m,  the  true  mean,  100  (1-a)  % 
of  the  time. 


Procedure 

Problem:  What  is  a  two-sided  100  (1  —  a)  % 
confidence  interval  for  the  true  mean  m? 

(1)  Choose  the  desired  confidence  level,  l  —  a 


Example 

Problem:  What  is  a  two-sided  95%  confidence 
interval  for  the  mean  thickness  in  the  lot? 
(Data  Sample  2-1) 

(1)  Let  1  -  a  =  .95 
«  =  .05 


C2)  Compute: 

X,  the  arithmetic  mean  (see  Paragraph 
2-1.2),  and 

s  .  In XX*  2 

\  «  (n  —  1) 


(2) 


X  =  .1260  inch 


s  =  0.00359  inch 
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Procedure 

(31  Look  up  t  ~  an  for  n  —  1  degrees  of 
freedom*  in  Table  A-4. 


(4j  Compute: 


Xu  —  X  t 


Xu  =  X  -  t 


y/n 


\/n 


(3) 

(4) 


Example 

t  =  t.tn  for  9  degrees  of  freedom 

-=  2.262 


Xu  =  .1260  + 


2.262  (.00359) 


VlO 

.1286  inch 

2.262  (.00359) 


-  1260  -  vM 
=  .1234  inch 


Conclude:  The  interval  from  XL  to  Xt,  is  a 
100  (1  —  a)  %  confidence  interval  for  the 
population  mean;  i.e.,  we  may  assert  with 
100  (1  —  a)  %  confidence  that  XL  <  m  <  Xu- 


Conclude:  The  interval  from  .1234  to  .1286  inch 
is  a  95%  confidence  interval  for  the  lot  mean; 
i.e.,  we  may  assert  with  95%  confidence  that 
.1234  inch  <  lot  mean  <  .1286  inch. 


2-1. 4.2  One-Sided  Confidence  Interval 

The  preceding  computations  can  be  used  to  make  another  kind  of  confidence  interval  state¬ 
ment.  We  can  say  that  100  (a/2)  %  of  the  time  the  entire  interval  in  Paragraph  2-1.4.1  will 
lie  above  the  true  mean  (i.e.,  Xu,  the  lower  limit  of  the  interval  will  be  larger  than  the  true 
mean) .  The  rest  of  the  time — namely  100  ( 1  —a/2)  %  of  the  time — XL  will  be  less  than  the 
true  mean.  Hence  the  interval  from  XL  to  +  oc  is  a  100(1  ~a/2)%  one-sided  confidence  in¬ 
terval  for  the  true  mean.  In  the  example,  Paragraph  2-1.4.1,  100(1  -a/2)  %  equals  97.5%. 
Thus,  either  of  two  open-ended  intervals,  “larger  than  .1234  inch,”  or  “less  than  .1286  inch” 
can  be  called  a  97.5%  one-sided  confidence  interval  for  the  population  mean. 

We  now  give  the  step-by-step  procedure  for  determining  a  one-sided  confidence  interval 
for  the  population  mean  corresponding  to  a  different  choice  of  confidence  level. 


*  In  A  Dictionary  of  Statistical  Terms, *'t  we  find  the  following,  under  the  phrase 
“degrees  of  freedom” : 

“This  term  is  used  in  statistics  in  slightly  different  senses.  It  was  introduced  by 
Fisher  on  the  analogy  of  the  idea  of  degrees  of  freedom  of  a  dynamical  system,  that 
is  to  say  the  number  of  independent  coordinate  values  which  are  necessary  to  deter¬ 
mine  it.  In  this  sense  the  degrees  of  freedom  of  a  set  of  observations  (which  ex 
hypothesi  are  subject  to  sampling  variation)  is  the  number  of  values  which  could  be 
assigned  arbitrarily  within  the  specification  of  the  system;  for  example,  in  a  sample 
of  constant  size  n  grouped  into  k  intervals  there  are  k  —  1  degrees  of  freedom  be¬ 
cause,  if  k  —  1  frequencies  are  specified,  the  other  is  determined  by  the  total  size  n; 

A  sample  of  n  variate  values  is  said  to  have  n  degrees  of  freedom,  whether  the 
variates  are  dependent  or  not,  and  a  statistic  calculated  from  it  is,  by  a  natural  exten¬ 
sion,  also  said  to  have  n  degrees  of  freedom.  But,  if  k  functions  of  the  sample  values 
arc  held  constant,  the  number  of  degrees  of  freedom  is  reduced  by  k.  For  example, 

the  statistic  (*»  —  *)•  where  i  is  the  sample  mean,  is  said  to  have  to  —  1 

t“  I 

degrees  of  freedom.  ...” 

In  this  example,  s2  =  (Xl  ■'’)*/(«  -  1)  and  has  "n  —  1"  degrees  of  freedom. 
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Procedure 

Problem:  What  is  a  one-sided  100  (1  —  a)  % 
confidence  interval  for  the  true  mean? 


Example 

Problem:  What  is  a  value  which  we  expect,  with 
99%  confidence,  to  be  exceeded  by  the  lot 
mean?  (Alternatively,  what  is  a  value  which  we 
expect,  with  99%  confidence,  to  exceed  the  lot 
mean?)  (Data  Sample  2-1) 


(1) 

Choose  the  desired  confidence  level,  1  —  a. 

(1) 

Let  1  -  «  =  .99 

a  =  0.01. 

(2) 

Compute: 

X 

8 

(2) 

X  =  .1260  inch 

8  =  0.00369  inch 

(3) 

Look  up  t  =  1 for  n  —  1  degrees  of  free¬ 
dom  in  Table  A-4. 

(3) 

t  =  f  .»*  for  9  degrees  of  freedom 
=  2.821 

(4) 

Compute: 

(4) 

(or  compute 

*£  =  .1260  (2-821)  (-00369) 

vlo 

=  .1228 

(or  Xv  =  .1292) 

Conclude:  We  are  100  (1  —  a)  %  confident  that 
the  lot  mean  to  is  greater  than  X  'h  (or,  we  are 
100  (1  —  a)  %  confident  that  the  lot  mean  to 
is  less  than  Xu),  i.e.,  we  may  assert  with 
100  (1  — a)  %  confidence  that  to  >  XL  (or, 
that  to  <  Xv). 


Conclude:  We  are  99%  confident  that  the  lot 
mean  is  greater  than  .1228  inch  (or,  we  are  99% 
confident  that  the  lot  mean  is  less  than  .1292 
inch),  i.e.,  we  may  assert  with  99%  confidence 
that  mean  thickness  in  lot  >  .1228  inch  (or, 
that  mean  thickness  in  lot  <  .1292  inch). 


2-1.5  CONFIDENCE  INTERVAL  ESTIMATES  WHEN  WE  HAVE  PREVIOUS  KNOWLEDGE 
OF  THE  VARIABILITY 

We  have  assumed  in  the  previous  paragraph  (2-1.4)  that  we  had  no  previous  information 
about  the  variability  of  performance  of  items  and  were  limited  to  using  an  estimate  of  vari¬ 
ability  obtained  from  the  sample  at  hand.  Suppose  that  in  the  case  of  the  mica  washers  we 
had  taken  samples  many  times  previously  from  the  same  process  and  found  that,  although 
each  lot  had  a  different  average,  there  was  always  essentially  the  same  amount  of  variation 
within  a  lot.  We  would  then  be  able  to  take  <r,  the  standard  deviation  of  the  lot,  as  known  and 
equal  to  the  value  indicated  by  this  previous  experience.  This  assumption  should  not  be  made 
casually,  but  only  when  warranted  after  real  investigation  of  the  stability  of  the  variation 
among  samples,  using  control  chart  techniques. 

The  procedure  for  computing  these  confidence  intervals  is  simple.  In  the  procedures  of 
Paragraph  2-1.4,  merely  replace  s  by  cr  and  t  by  z  and  the  formulas  remain  the  same.  Values 
of  z  are  given  in  Table  A-2.  Note  that  tr  for  an  infinite  number  of  degrees  of  freedom 
(Table  A-4)  is  exactly  equal  to  zP.  The  following  procedure  is  for  the  two-sided  confidence 
interval. 
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Procedure 


Example 


Problem:  Find  a  two-sided  100  (1  —  a)  %  confi¬ 
dence  interval  for  the  lot  mean,  using  known  <r. 

(1)  Choose  the  desired  confidence  level,  1  —  a. 

(2)  Compute: 

X 

(3)  Look  up  2  =  Zi—a/x  in  Table  A-2. 

(4)  Compute: 

Xu  =  X  +  2  — y=- 
V» 

XI  =  *  ~ 


Problem:  What  is  a  two-sided  95%  confidence 
interval  for  the  lot  mean?  (Data  Sample  2-1; 
and  a  is  known  to  equal  .0040  inch.) 

(1)  Let  1  -  «  =  .95 

a  =  .05 

(2) 

X  ~  .1260  inch 

(3)  2  =  2.»  ti 

=  1.960 

(4) 

Xv  «  .1260  +  1-960 

V10 

=  .1285 
Xi  =  .1235 


Conclude:  The  interval  from  Xi  to  Xu  is  a  Conclude:  The  interval  from  .1235  to  .1285  inch 
100  (1  —  a)  %  confidence  interval  for  the  lot  is  a  95%  confidence  interval  for  the  lot  mean, 
mean. 


Discussion:  When  the  value  of  <r,  the  standard  deviation  in  the  population,  is  known.  Pro¬ 
cedure  2-1.6  should  always  be  used  in  preference  to  Procedure  2-1.4,  which  is  independent 
of  our  knowledge  of  <r.  When  available,  Procedure  2-1.5  (tr  known)  will  usually  lead  to  a 
confidence  interval  for  the  population  mean  that  is  narrower  than  the  confidence  interval 
that  would  have  been  obtained  by  Procedure  2-1.4  (<r  unknown).  This  is  the  case  for  our 
illustrative  examples  based  on  Data  Sample  2-1,  but  the  difference  is  very  slight  because 
<t  and  s  were  both  very  small — only  0.03%  of  the  mean. 

Whatever  level  of  confidence  is  chosen,  the  t  value  required  for  the  application  of  Proce¬ 
dure  2-1.4  (tr  unknown)  will  always  be  larger  than  the  corresponding  z  value  required  for 
Procedure  2-1.6  (cr  known).  This  is  evident  from  Table  A-4.  For  very  small  samples,  the 
difference  can  be  considerable.  Nevertheless,  it  can  happen,  as  a  result  of  unusual  sampling 
fluctuations,  that  the  value  of  <t  obtained  in  a  particular  sample  is  so  small  in  comparison  to 
cr  that,  if  Procedure  2-1.4  (cr  unknown)  were  used,  the  resulting  confidence  interval 
would  be  narrower  than  the  confidence  interval  given  by  Procedure  2-1.5  (cr  known).  This 
would  have  been  the  case,  for  instance,  if  Data  Sample  2-1  had  yielded  an  s  less  than 
1.960 (0.0040) /2.262  =  0.00347.  With  samples  of  size  10  (i.e.,  9  degrees  of  freedom  for  a), 
the  probability  of  such  an  occurrence  is  about  one  in  three.  In  such  a  case,  however,  one 
must  NOT  adopt  the  confidence  interval  corresponding  to  Procedure  2-1.4  (cr  unknown)  be¬ 
cause  it  is  narrower.  To  choose  between  Procedure  2-1.4  (cr  unknown)  and  Procedure  2-1.5 
(<r  known),  when  the  value  of  cr  IS  known,  by  selecting  the  one  which  yields  the  narrower 
confidence  interval  in  each  instance,  would  result  in  a  level  of  confidence  somewhat  lower 
than  claimed. 
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2-2  ESTIMATING  VARIABILITY  OF  PERFORMANCE  FROM  A  SAMPLE 


2-2.1  GENERAL 

We  take  the  standard  deviation  of  per¬ 
formance  in  the  population  as  our  measure  of 
the  characteristic  variability  of  performance. 
Presented  here  are  various  ways  of  estimat¬ 
ing  the  population  standard  deviation,  in¬ 
cluding: 

(a)  single-value  estimates; 

(b)  confidence-interval  estimates  based 
on  random  samples  from  the  population; 
and, 

(c)  techniques  for  estimating  the  popula¬ 
tion  standard  deviation  when  no  appropriate 
random  samples  are  available. 

The  first  two  procedures  are  illustrated  by 
application  to  the  following  data. 

Data  Sample  2-2 — Burning  Time  of 
Rocket  Powder 


a  sufficient  number  of  decimal  places.  If 
too  few  places  are  carried,  the  subtractions 
involved  may  result  in  a  loss  of  significant 
figures  in  s2.  Excessive  rounding  may  even 
lead  to  a  negative  value  for  s2.  The  formula 
recommended  for  computational  purposes  is 
to  be  preferred  on  this  account  because  only 
otic  subtraction  is  involved ;  and  with  a  desk 
calculator  one  usually  can  retain  all  places  in 
the  computation  of  ZXf  and  (sZ<)2. 

We  take 

\  n  (n  —  1) 

as  our  estimate  of  <r,  the  population  standard 
deviation. 

Example:  Using  Data  Sample  2-2,  2.X7  = 
27987.54,  ZX>  =  519.8,  and  thus  s2  =  107.593; 
and  s  =  10.37  seconds.* 


Form:  n  independent  measurements  A',,  X.,, 
.  .  .,  X„  selected  at  random  from  a 
much  larger  group. 

Example:  Ten  unit  amounts  of  rocket  pow¬ 
der  selected  at  random  from  a  large 
lot  were  tested  in  a  chamber  and 
their  burning  times  recorded  as  fol¬ 
lows  (seconds) : 


50.7 

69.8 

54.9 

53.4 

54.3 

66.1 

44.8 

48.1 

42.2 

35.5 

2-2.2  SINGLE  ESTIMATES 
2-2.2. T  s2  and  s 


The  best  estimate  of  <x-,  the  variance  of  a 
normal  population,  is: 


n 


For  computational  purposes,  we  usually  find 
it  more  convenient  to  use  the  following  for¬ 
mula: 


ns?' 
2  _  •“* 


s2  = 


n  (n  —  1) 

The  formulas  are  algebraically  identical. 
With  any  formula,  it  is  important  to  carry 


2-2.2.2  The  Sample  Range  as  an  Estimate  of 
the  Standard  Deviation 

The  range  of  n  observations  is  defined  as 
the  difference  between  the  highest  and  the 

TABLE  2-1.  TABLE  OF  FACTORS  FOR  CONVERT¬ 
ING  THE  RANGE  OF  A  SAMPLE  OF  n  TO  AN 
ESTIMATE  OF  <r,  THE  POPULATION  STANDARD 
DEVIATION.  ESTIMATE  OF  a  =  RANGE/d„ 


Size  of 
Sample 

a 

1 

df> 

y/n 

[See  Nofe] 

2 

1.128 

.8865 

1.414 

3 

1.693 

.5907 

1.732 

4 

2.059 

.4857 

2.000 

5 

2.326 

.4299 

2.236 

6 

2.534 

.3946 

2.449 

7 

2.704 

.3698 

2.646 

8 

2.847 

.3512 

2.828 

9 

2.970 

.3367 

3.000 

10 

3.078 

.3249 

3.162 

12 

3.258 

.3069 

3.464 

16 

3.532 

.2831 

4.000 

Note:  d„  is  approximately  equal  to  /n  for  8  ^  n 
^  10.  Thus,  for  small  n  a  quick  estimate  of  o  can 
be  obtained  by  dividing;  the  range  by  /' n 


*  In  a  final  report,  values  of  e  should  be  rounded 
to  two  significant  figures,  but  as  a  basis  for  further 
calculations  it  is  advisable  to  retain  one  or  two  addi¬ 
tional  figures.  For  fuller  explanation,  see  Chapters 
22  and  23,  Section  4,  AMCP  706-118. 
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lowest  of  the  n  values.  For  small  samples,  As  the  sample  size  gets  larger,  the  range  is 

the  sample  range  is  a  reasonably  efficient  not  only  troublesome  to  calculate,  but  is  a 

substitute  for  s  as  an  estimator  of  the  stand-  very  inefficient  estimator  of  cr.  Table  2-1 

ard  deviation  of  a  normal  population — not  as  gives  the  factors  which  convert  from  ob- 

efficient  as  s,  but  easier  to  calculate.  Using  served  range  in  a  sample  of  n  observations 

the  range  is  particularly  valuable  for  a  to  an  estimate  of  population  standard  devia- 

“quick  look”  at  data  from  small  samples.  tion  cr. 


2-2.3  CONFIDENCE  INTERVAL  ESTIMATES* 

2-2.3. 1  Two-Sided  Confidence  Interval  Estimates 

We  are  interested  in  determining  an  interval  which  we  may  confidently  expect  to  bracket 
the  true  value  of  the  standard  deviation  of  a  normal  population. 

Procedure  Example 

Problem:  What  is  a  two-sided  100  (1  —  a)  %  Problem:  What  is  a  95%  confidence  interval  for 
confidence  interval  for  <r?  cr,  the  standard  deviation  of  burning  time  in  the 

lot  of  powder?  (Data  Sample  2-2) 

(1)  Choose  the  desired  confidence  level,  1  --  «.  (1)  Let  1  —  a  =  .95 

a  =  .05 

(2)  Compute:  (2) 

S  =  lnxXLz.  (^XA'  S  =  10.37  seconds 

\  n  (n  —  1) 

(3)  Look  up  Bu  and  Bc  for  n  —  1  degrees  of  (3)  For  9  degrees  of  freedom, 

freedom  in  Table  A-20.  BL  —  .6657 

Bv  =  1.746 

(4) 

sL  =  (10.37)  (.6657) 

=  6.90  seconds 
Sv  =  (10.37)  (1.746) 

=  18.11  seconds 

Conclude:  The  interval  from  sL  to  s„  is  a  two-  Conclude:  The  interval  from  6.90  to  18.11  is  a 
sided  100  (1  —  a)  %  confidence  interval  estimate  two-sided  95%  confidence  interval  for  <r;  i.e.,  we 
for  a\  i.e.,  we  may  assert  with  100  (1  —  a)  %  may  assert  with  95%  confidence  that  6.90 
confidence  that  sL  <  a  <  sv.  seconds  <  <r  <  18.11  seconds. 


2-2.3.2  One-Sided  Confidence  Interval  Estimates 

In  some  applications  we  are  not  particularly  interested  in  placing  both  an  upper  and  a 
lower  bound  on  cr,  but  only  in  knowing  whether  the  variability  is  excessively  large  (or, 
exceptionally  small) .  We  would  like  to  make  a  statement  such  as  the  following:  We  can  state 

*  The  reader  who  is  not  familiar  with  the  meaning  and  interpretation  of  confidence  intervals  should  refer 
to  Chapter  1,  and  to  Paragraph  2-1.3  of  this  chapter.  The  remarks  of  Paragraph  2-1.3  concerning  confidence 
intervals  for  the  average  carry  over  to  confidence  intervals  for  a  measure  of  variability. 


(4)  Compute: 
sL  =  Bls 

sv  =  BLs 
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with  100(1  -  a)  %  confidence  that  the  variability  as  measured  by  <r  is  less  than  some  value 
si  computed  from  the  sample.  Similarly,  but  not  simultaneously,  we  may  wish  to  state  with 
100(1  -  a)  %  confidence  that  cr  is  greater  than  some  value  si.  Either  statement  is  a  one¬ 
sided  confidence  interval  estimate. 


Procedure 

Problem:  What  is  a  value  si,  such  that  we  may 
have  100  (1  —  a)  %  confidence  that  a  is  less 
than  s(,1 


Example 

Problem:  What  is  a  value  &{,  such  that  we  have 
95%  confidence  that  <r  is  less  than  s{,t  (Data 
Sample  2-2) 


(1)  Choose  the  desired  confidence  level,  1  —  a. 

(2)  Compute: 

s 

(3)  Look  up  for  n  —  1  degrees  of  freedom 
in  Table  A-21. 

(4)  Compute: 

s{j  —  A  1 — a  S 

(5)  With  100  (1  —  a)  %  confidence  we  can  as¬ 
sert  that  a  is  less  than  sj. 


(1)  Let  1  —  a  =  .95 

a  =  .05 

(2) 

s  -  10.37  seconds 

(3)  For  9  degrees  of  freedom, 

A  =  1.645 

(4) 

sv  =  (1.645)  (10.37) 

=  17.06  seconds 

(5)  We  are  95%  confident  that  the  variability 
as  measured  by  a  is  less  than  si  =  17.06 
seconds. 


Should  a  lower  bound  to  cr  be  desired,  follow  Procedure  2-2.3.2  with  si  and  replaced 
by  s'L  and  Aa,  respectively.  Then  it  can  be  asserted  with  100  (1  -  a)  %  confidence  that  cr  >  si. 


2-2.4  ESTIMATING  THE  STANDARD  DEVIA¬ 
TION  WHEN  NO  SAMPLE  DATA  ARE 
AVAILABLE 


(a)  Estimate  two  values  a,  and  b,  between 
which  you  expect  99.7%  (almost  all)  of  the 
individuals  to  be.  Then,  estimate 


It  is  often  necessary  to  have  some  idea  of 
the  magnitude  of  the  variation  of  some  char¬ 
acteristic  as  measured  by  cr,  its  standard 
deviation  in  the  population.  In  planning  ex¬ 
periments,  for  example,  the  sample  size  re¬ 
quired  to  meet  certain  requirements  is  a 
function  of  cr.  In  almost  any  situation,  one 
can  get  at  least  a  very  rough  estimate  of  cr. 
The  minimum  necessary  information  in¬ 
volves  the  form  of  the  distribution  and  the 
spread  of  values.  For  example,  if  the  values 
of  the  individual  items  can  be  assumed  to 
follow  a  normal  distribution,  then  either  of 
the  following  rules  can  be  used  to  get  an 
estimate  of  cr: 


<7  as 


(b)  Estimate  two  values  a,  and  b,  between 
which  you  expect  95%  of  the  individuals  to 

be.  Then,  estimate  a  as  k*  .1 


If  the  distribution  concerned  cannot  be  as¬ 
sumed  to  be  normal  but  can  be  assumed  to 
follow  one  of  the  top  four  forms  in  Figure 
2-1,  then  the  standard  deviation  may  be  esti¬ 
mated  as  indicated  in  the  figure.  This  figure 
also  illustrates  the  distribution  and  rules  for 
(a)  and  (b)  above. 
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DISTRIBUTION  STANDARD 

DEVIATION 

b  ~  a 

_  3.5 

a  l 


4 


Figure  2-1.  The  standard  deviation  of  some 
simple  distributions. 

Adapted  with  permiwlon  from  Soma  Thaont  of  Sampling,  by 
W.  Edward*  Demin*,  Copyright,  1660.  John  Wiley  A  Son*.  Inc. 


2-3  NUMBER  OF  MEASUREMENTS  REQUIRED  TO  ESTABLISH 
THE  MEAN  WITH  PRESCRIBED  ACCURACY 


2-3.1  GENERAL 

In  planning  experiments,  we  may  need  to 
know  how  many  measurements  or  how  large 
a  sample  to  take  in  order  to  determine  the 
mean  of  some  distribution  with  prescribed 


accuracy.  Suppose  we  are  willing  to  allow 
a  margin  of  error  d,  and  a  risk  a  that  our 
estimate  of  m  will  be  off  by  an  amount  d  or 
greater.  Since  the  sampling  distribution  of 
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X  is  “normal”  to  a  good  approximation  for 
samples  of  four  or  more  measurements  from 
almost  every  population  distribution  likely 
to  be  met  in  practice,  we  can  ascertain  the 
required  sample  size  n  if  we  have  an  avail¬ 
able  estimate  s  of  cr,  or  if  we  are  willing  to 
assume  that  we  know  cr.  If  we  have  not 
made  an  estimate  or  are  unwilling  to  assume 
a  value  for  cr,  then  we  must  use  a  two-stage 
sample.  The  two-stage  method  will  usually 
result  in  a  smaller  total  sample  size.  In  the 
two-stage  method,  we  must  start  by  guessing 
a  value  of  cr,  but  the  end  results  do  not  de¬ 
pend  upon  how  good  or  bad  is  the  guess. 


Sometimes  we  may  have  available  to  us 
one  or  more  samples  from  the  population  of 
interest,  from  which  we  can  derive  an  esti¬ 
mate  s  of  cr  based  on  v  degrees  of  freedom. 
Other  times  we  may  have  one  or  more  sam¬ 
ples  from  some  other  population  that  has  the 
same  standard  deviation  as  the  population 
of  interest,  but  possibly  a  different  mean. 
Again,  we  can  derive  an  estimate  s  of  cr 
based  on  v  degrees  of  freedom.  In  either 
case,  we  can  utilize  this  preliminary  estimate 
of  cr  to  determine  the  sample  size  n  required 
to  estimate  the  mean  of  the  population  of 
interest  with  prescribed  accuracy. 


2-3.2  ESTIMATION  OF  THE  MEAN  OF  A  POPULATION  USING  A  SINGLE  SAMPLE 


Procedure 

Problem:  We  wish  to  know  the  sample  size  re¬ 
quired  to  ascertain  the  mean  m  of  a  population. 
We  are  willing  to  take  a  risk  a  that  our  estimate 
is  off  by  d  or  more.  There  is  available  an  esti¬ 
mate  s  of  the  population  standard  deviation  a, 
based  on  v  degrees  of  freedom. 

(1)  Choose  d,  the  allowable  margin  of  error, 
and  a,  the  risk  that  our  estimate  of  m  will 
be  off  by  d  or  more. 

(2)  Look  up  for  v  degrees  of  freedom  in 
Table  A-4. 

(3)  Compute: 


Example 

Problem:  We  wish  to  know  the  average  thick¬ 
ness  of  the  washers  in  a  given  lot.  We  are  willing 
to  take  a  risk  that  5  times  in  100  the  error  in  our 
estimate  will  be  0.002  inch  or  more.  From  a 
sample  from  another  lot  we  have  an  estimate  of 
the  population  standard  deviation  of  s  =  .00359 
with  9  degrees  of  freedom. 

(1)  Let  d  -  0.002  inch 

a  =  .05 

(2)  t  -  (.» 7j  for  9  degrees  of  freedom 

=  2.262. 

(3) 


t1si 

d* 


Conclude:  If  we  now  compute  the  mean  X  of  a 
random  sample  of  size  n  from  the  population  we 
may  have  100  (1  -  a)  %  confidence  that  the 
interval  Ar  —  d  to  X  +  d  will  include  the  popu¬ 
lation  mean  m. 

If  we  know  cr,  or  assume  some  value  for 
procedure.  Values  of  z 


(2.262)*  (.00359)*  _ 

71  =  (^02F  "  =  165 

=  17  (conventionally  rounded  up  to 
the  next  integer.) 

Conclude:  We  may  conclude  that  if  we  now  com¬ 
pute  the  mean  X  of  a  random  sample  of  size 
n  =  17  from  the  lot  of  washers,  we  may  have 
95%  confidence  that  the  interval  X  -  .002 
to  X  4-  .002  will  include  the  lot  mean. 

by  Zi-a/i  in  the  above 


cr,  replace  s  by  cr  and  £,_«/. 
are  given  in  Table  A-2. 


2-3.3  ESTIMATION  USING  A  SAMPLE  WHICH  IS  TAKEN  IN  TWO  STAGES 

It  is  possible  that  we  do  not  have  a  good  estimate  of  cr,  the  standard  deviation  of  the  popu¬ 
lation.  When  the  cost  of  sampling  is  high,  rather  than  take  a  larger  sample  than  is  really 
necessary,  we  might  prefer  to  take  the  sample  in  two  stages.  The  method  (sometimes  called 
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Stein’s  method)  goes  roughly  as  follows  :  Make  a  guess  for  the  value  of  <x.  From  this  deter¬ 
mine  frj  the  size  of  the  first  sample.  The  first  sample  will  provide  an  estimate  s  of  the  popula¬ 
tion  standard  deviation.  Use  this  value  of  s  to  determine  how  large  the  second  sample 
should  be. 


Procedure 

Problem:  We  wish  to  know  the  sample  size  re¬ 
quired  to  ascertain  the  mean  m  of  a  population. 
We  are  willing  to  take  a  risk  a  that  our  estimate 
is  off  by  d  or  more  units. 


Example 

Problem:  We  have  a  large  lot  of  devices,  and 
wish  to  determine  the  average  of  some  property. 
We  are  willing  to  take  a  risk  of  .05  of  the  esti¬ 
mate  being  in  error  by  30  units. 


(1)  Choose  d,  the  allowable  margin  of  error, 
and  a,  the  risk  that  our  estimate  of  m  will 
be  off  by  d  or  more. 

(2)  Let  a'  be  the  best  possible  guess  for  the 
value  of  <r,  the  standard  deviation  of  the 
population  (see  Paragraph  2-2.4). 

(3)  Look  up  in  Table  A-2. 

(4)  Compute: 

•'  -  )' 

n'  is  the  first  estimate  of  the  total  sample 
size  required. 

(5)  Choose  n,  the  size  of  the  first  sample,  th 
should  be  considerably  less  than  n\  (If  the 
guessed  value  of  a  is  too  large,  this  will  pro¬ 
tect  us  against  a  first  sample  which  is 
already  larger  than  we  need.)  A  rough  rule 
might  be  to  make  nt  >  30  unless  «'  <  60, 
in  which  case  let  n,  be  somewhere  between 
.5 n'  and  .In'. 

(6)  Make  the  necessary  observations  on  the 
sample  of  n„  Compute  Sj,  the  standard 
deviation. 

(7)  Look  up  for  nx  —  1  degrees  of  freedom 
in  Table  A-4. 

(8)  Compute 


n  is  the  total  required  sample  size  for  the  first 
and  second  samples  combined.  We  then 
require  a  second  sample  size  of  n ,  =  n  —  ny. 


(1)  Let  d  =  30 
«  =  .05 


(2)  From  our  knowledge  of  similar  devices  our 
best  estimate  of  <r  is  200  units. 


(3) 

z.m 

=  1.960 

(4) 

n' 

(1.960)*  (200) 

(30)* 

=  170.7 

(5) 

Let  nt 

=  50 

(6)  From  tests  on  50  devices  chosen  at  random, 
Si  =  160  units. 


(7)  t  =  t.t 7*  for  49  degrees  of  freedom 

=  2.01. 

(8) 

„  _  (2.01) » (160) » 

(30) » 

=  114.9 
=  115 

n,  =  116  -  60 
=  65 
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If  now  we  obtain  the  second  sample  of  size  n2  and  compute  the  mean  X  of  the  total  sample 
of  size  n  =  m  4-  »*,  we  may  have  100  (1  —  a)  %  confidence  that  the  interval  X  —  d  to  X  +  d 
will  include  the  population  mean  to. 

2-4  NUMBER  OF  MEASUREMENTS  REQUIRED  TO  ESTABLISH 
THE  VARIABILITY  WITH  STATED  PRECISION 

We  may  wish  to  know  the  size  of  sample  required  to  estimate  the  standard  deviation  with 
certain  precision.  If  we  can  express  this  precision  as  a  percentage  of  the  true  (unknown) 
standard  deviation,  we  can  use  the  curves  in  Figure  2-2. 


5  6  e  10  20  30  40  50 

P% 

Figure  2-2.  Number  of  degrees  of  freedom  required  to 
estimate  the  standard  deviation  within  P%  of  its 
true  value  v'ith  confidence  coefficient  y. 

Adapted  with  permission  from  Journal  of  the  American  Statistical  AeeoeuUion ,  Vol.  46  (1060).  p.  268,  from  article  entitled  "Sample 
Site  Required  for  Estimating  the  Standard  Deviation  a*  a  Percent  of  1U  True  Value"  by  J.  A.  Greenwood  and  M.  M.  Sandomlre, 
m * r  tfraphin*  la  adapted  with  permission  from  Statistic*  Manual  by  E.  L.  Crow,  F.  A.  Davis,  and  M.  W.  lfaxfleld, 
NAVORD  Report  3869.  NOTS  048,  U.  S.  Naval  Ordnance  Teat  Station,  China  Lake,  Calif,.  1966.  (Reprinted  by  Dover  Publications. 
Inc.,  New  York,  N.Y.,  1960.) 
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Procedure 

Problem:  If  we  are  to  make  a  simple  series  of 
measurements,  how  many  measurements  are  re¬ 
quired  to  estimate  the  standard  deviation  with¬ 
in  P  percent  of  its  true  value,  with  prescribed 
confidence? 

(1)  Specify  P,  the  allowable  percentage  devia¬ 
tion  of  the  estimated  standard  deviation 
from  its  true  value. 

(2)  Choose  y,  the  confidence  coefficient. 

(3)  In  Figure  2-2,  find  P  on  the  horizontal 
scale,  and  use  the  curve  for  the  appropriate 
y.  Read  on  the  vertical  scale  the  required 
degrees  of  freedom. 

(4)  For  a  simple  series  of  measurements,  the 
required  number  is  equal  to  one  plus  the 
degrees  of  freedom. 


Exampi* 

Problem:  How  large  a  sample  would  be  required 
to  estimate  the  standard  deviation  within  20% 
of  its  true  value,  with  confidence  coefficient 
equal  to  0.95? 

(1)  Let  P  =  20% 

(2)  Let  y  =  .95 

(3)  For  y  -  .95y  P  =  20%,  the  required  de¬ 
grees  of  freedom  equals  46. 

(4)  n  -  46  +  1 

=  47 


?-5  STATISTICAL  TOLERANCE  LIMITS 


2-5.1  GENERAL 

Sometimes  we  are  more  interested  in  the 
approximate  range  of  values  in  a  lot  or  popu¬ 
lation  than  we  are  in  its  average  value.  Sta¬ 
tistical  tolerance  limits  furnish  limits  be¬ 
tween,  above,  or  below  which  we  confidently 
expect  to  find  a  prescribed  proportion  of  in¬ 
dividual  items  of  the  population.  Thus,  we 
might  like  to  be  able  to  give  two  values  A 
and  B  between  which  we  can  be  fairly  cer¬ 
tain  that  at  least  a  proportion  P  of  the  popu¬ 
lation  will  lie,  (two-sided  limits),  or  a  value 
A  above  which  at  least  a  proportion  P  will 
lie,  (one-sided  limit) . 


Thus  for  the  data  on  thickness  of  mica 
washers  (Data  Sample  2-1),  we  could  give 
two  thickness  values,  stating  with  chosen 
confidence  that  a  proportion  P  (at  least)  of 
the  washers  in  the  lot  have  thicknesses  be¬ 
tween  these  two  limits.  We  call  the  confi¬ 
dence  coefficient  y,  and  it  refers  to  the  pro¬ 
portion  of  the  time  that  our  method  will 
result  in  correct  statements.  If  a  normal  dis¬ 
tribution  can  be  assumed,  use  the  procedures 
of  Paragraphs  2-5.2  and  2-5.3 ;  otherwise  use 
the  procedures  of  Paragraph  2-5.4. 


2-5.2  TWO-SIDED  TOLERANCE  LIMITS  FOR  A  NORMAL  DISTRIBUTION 

When  the  mean  m  and  standard  deviation  c  of  a  normally  distributed  quantity  are  known, 
symmetrical  limits  that  include  a  prescribed  proportion  P  of  the  distribution  are  readily 
obtained  by  adding  and  subtracting  za  cr  from  the  known  mean  m,  where  z„  is  read  from 
Table  A-2  with  a  =  $(P+ 1).  When  m  and  cr  are  not  known,  we  can  use  an  interval  of  the 
form  X  ±  Ks.  Since  both  X  and  s  will  vary  from  sample  to  sample  it  is  impossible  to 
determine  K  so  that  the  limits  X  ±  Ka  will  always  include  a  specified  proportion  P  of  the 
underlying  normal  distribution.  It  is,  however,  possible  to  determine  K  so  that  in  a  long 
series  of  samples  from  the  same  or  different  normal  distributions  a  definite  proportion  y  of 
the  intervals  X  ±  Ks  will  include  P  or  more  of  the  underlying  distribution  (s). 
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Procedure 

Problem:  We  would  like  to  state  two  limits 
between  which  we  are  100  y  percent  confident 
that  100  P  percent  of  the  values  lie. 

(1)  Choose  P,  the  proportion,  and  y,  the  confi¬ 
dence  coefficient. 

(2)  Compute  from  the  sample: 

X 

s 

(3)  Look  up  K  for  chosen  P  and  y  in  Table  A-6. 

(4)  Compute: 

Xir  —  X  +  Ks 

XL  =  X  -  Ks 

Conclude:  With  100  y  %  confidence  we  may  pre¬ 
dict  that  a  proportion  P  of  the  individuals  of  the 
population  have  values  between  XL  and  Xu. 


Example 

Problem:  We  would  like  to  state  thickness  limits 
between  which  we  are  95%  confident  that  90% 
of  the  values  lie  (Data  Sample  2-1). 

(1)  Let  P  =  .90 

7  =  .95 

(2) 

X  =  .1260  inch 
s  =  0.00359  inch 

(3)  K  =  2.839 

(4) 

Xu  =  .1260  +  2.839  (.00359) 

=  0.136  inch 

XL  =  .1260  -  2.839  (.00359) 

=  0.116  inch 

Conclude:  With  95%  confidence,  we  may  say 
that  90%  of  the  washers  have  thicknesses  be¬ 
tween  0.116  and  0.136  inch. 


2-5.3  ONE-SIDED  TOLERANCE  LIMITS  FOR  A  NORMAL  DISTRIBUTION 

Sometimes  we  are  interested  only  in  estimating  a  value  above  which,  or  below  which,  a 
proportion  P  (at  least)  will  lie.  In  this  case  the  one-sided  upper  tolerance  limit  will  be 
Xu  =  X  +  Ks;  and  XL  =  X  -  Ks  will  be  the  one-sided  lower  limit.  The  appropriate  values 
for  K  are  given  in  Table  A-7  and  are  not  the  same  as  those  of  Paragraph  2-5.2. 


Procedure 

Problem:  To  find  a  single  value  above  which  we 
may  predict  with  confidence  y  that  a  proportion 
P  of  the  population  will  lie. 

(1)  Choose  P  the  proportion  and  y,  the  confi¬ 
dence  coefficient. 

(2)  Compute: 

X 

s 

(3)  Look  up  K  in  Table  A-7  for  the  appropriate 
n,  7,  and  P. 

(4)  XL  -  X  -  Ks 


Example 

Problem:  To  find  a  single  value  above  which  we 
may  predict  with  90%  confidence  that  99%  of 
the  population  will  lie.  (Data  Sample  2-1). 

(1)  Let  P  =  .99 

7  -  .90 

(2) 

X  =  .1260  inch 
s  =  0.00359  inch 

(3)  K  (10,  .90,  .99)  =  3.532 


(4)  XL  =  .1260  -  3.532  (.00359) 

=  .1133  inch 

Thus  we  are  90%  confident  that  99%  of  the 
mica  washers  will  have  thicknesses  above 
. 113  inch. 
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Note:  Factors  for  some  values  of  n,  y,  and  P  not  covered  in  Table  A-7  may  be  found  in 
Sandia  Corporation  Monograph  SCR-L3(2’.  Alternatively,  one  may  compute  K  using  the  fol¬ 
lowing  formulas: 


k  = Zp  ~ 

a 


(where  z  can  be  found  in  Table  A-2) 


2-5.4  TOLERANCE  LIMITS  WHICH  ARE  INDE¬ 
PENDENT  OF  THE  FORM  OF  THE 
DISTRIBUTION 

The  methods  given  in  Paragraphs  2-5.2 
and  2-5.3  are  based  on  the  assumption  that 
the  observations  come  from  a  normal  distri¬ 
bution.  If  the  distribution  is  not  in  fact 
normal,  then  the  effect  will  be  that  the  true 
proportion  P  of  the  population  between  the 
tolerance  limits  will  vary  from  the  intended 
P  by  an  amount  depending  on  the  amount  of 
departure  from  normality.  If  the  departure 
from  normality  is  more  than  slight  we  can 
use  a  procedure  which  assumes  only  that  the 
distribution  has  no  discontinuities.  The  tol¬ 
erance  limits  so  obtained  will  be  substantially 
wider  than  those  assuming  normality. 

2-5.4. T  Two-Sided  Tolerance  Limits 
(Distribution-Free) 

Table  A-30  gives  values  (r,  s)  such  that 
we  may  assert  with  confidence  at  least  y  that 
100P%  of  a  population  lies  betwreen  the  rth 
smallest  and  the  s'h  largest  of  a  random  sam¬ 
ple  of  n  from  that  population.  For  example, 
from  Table  A-30  with  y  —  .95,  P  =  .75,  and 
n  —  60,  we  may  say  that  if  we  have  a  sample 


of  n  -  60,  then  we  may  have  a  confidence  of 
at  least  y  =  .95  that  100P%  =  75%  of  the 
population  will  lie  between  the  fifth  largest 
(s  =  5)  and  the  fifth  smallest  (r  =  5)  of  the 
sample  values.  That  is,  if  we  were  to  take 
many  random  samples  of  60,  and  take  the 
fifth  largest  and  fifth  smallest  of  each,  we 
should  expect  to  find  that  at  least  95%  of  the 
resulting  intervals  would  contain  75%  of  the 
population. 

Table  A-32  may  be  useful  for  sample  sizes 
of  n  ^  100.  This  table  gives  the  confidence 
y  with  which  we  may  assert  that  100 P%  of 
the  population  lies  between  the  largest  and 
smallest  values  of  the  sample. 

2-5.4. 2  One-Sided  Tolerance  llmlti 
(Distribufion-Free) 

Table  A-31  gives  the  largest  value  of  m 
such  that  we  may  assert  with  confidence  at 
least  y  that  100P%  of  a  population  lies  be¬ 
low  the  m,h  largest  (or  above  the  m,h  small¬ 
est)  of  a  random  sample  of  n  from  that  pop¬ 
ulation.  For  example,  from  Table  A-31  with 
y  —  .95,  P  =  .90,  and  n  =  90,  we  may  say 
that  we  are  95%  confident  that  90%  of  a 
population  will  lie  below  the  fifth  largest 
value  of  a  sample  of  size  n  —  90. 
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CHAPTER  3 

COMPARING  MATERIALS  OR  PRODUCTS  WITH  RESPECT  TO 
AVERAGE  PERFORMANCE 


3-1  GENERAL  REMARKS  ON  STATISTICAL  TESTS 


One  of  the  most  frequent  uses  of  statistics 
is  in  testing  for  differences.  If  we  wish  to 
know  whether  a  treatment  applied  to  a  stand¬ 
ard  round  affects  its  muzzle  velocity,  we  may 
conduct  an  experiment  and  apply  a  statisti¬ 
cal  test  to  the  experimental  results  to  see 
whether  we  would  be  justified  in  concluding 
that  there  is  a  difference  between  the  per¬ 
formance  of  treated  and  untreated  rounds. 
In  another  case,  two  manufacturing  proc¬ 
esses  may  be  available — process  A  is  cheaper 
and  therefore  preferable  unless  process  B  is 
demonstrated  to  be  superior  in  some  respect. 
Again,  we  apply  a  statistical  test  to  the  ex¬ 
perimental  results  to  see  whether  process  B 
has  demonstrated  superiority. 

Ordinarily,  the  statistical  test  applied  to 
the  results  observed  on  a  sample  will  point 
the  way  to  decision  between  a  pair  of  alter¬ 
natives.  For  some  tests,  the  two  alternative 
decisions  will  be  formally  stated  as  follows: 

(a)  There  is  a  difference  between  the 
(population)  averages  of  two  materials, 
products,  processes,  etc. 

(b)  No  difference  has  been  demonstrated. 

In  other  cases,  the  formal  statement  of  the 
two  alternative  decisions  will  be : 

(a)  The  (population)  average  of  product 
A  is  greater  than  that  of  product  B. 

(b)  We  have  no  reason  to  believe  that  the 
(population)  average  of  product  A  is  greater 
than  that  of  product  B. 

In  this  Chapter  and  others,  we  shall  con¬ 
sider  a  number  of  statistical  tests  of  differ¬ 
ences.  The  application  of  each  statistical 


test  will  result  in  making  one  of  two  deci¬ 
sions,  as  in  the  pairs  given.  In  each  case  the 
pair  of  alternative  decisions  is  chosen  before 
the  data  are  observed — this  is  important,! 

Since  we  ordinarily  obtain  information  on 
one  or  both  of  the  products  by  means  of  a 
sample,  we  may  sometimes  make  an  errone¬ 
ous  decision.  However,  the  chance  of  making 
the  wrong  decision  can  be  reduced  by  in¬ 
creasing  the  number  of  observations.  There 
are  two  ways  in  which  we  can  make  a  wrong 
decision : 

(a)  When  we  conclude  that  there  is  a  dif¬ 
ference  where  in  fact  there  is  none,  we  say 
that  we  make  an  Error  of  the  First  Kind; 

(b)  When  we  fail  to  find  a  difference  that 
really  exists,  then  we  say  that  we  make  an 
Error  of  the  Second  Kind. 

In  any  particular  case,  we  never  can  be  abso¬ 
lutely  sure  that  the  correct  decision  has  been 
made,  but  w.e  can  know  the  probability  of 
making  either  type  of  error. 

The  probability  of  making  an  Error  of  the 
First  Kind  is  usually  denoted  by  a ;  and  the 
probability  of  making  an  Error  of  the  Sec¬ 
ond  Kind  is  denoted  by  (3.  The  ability  of  a 
given  statistical  test  to  detect  a  difference 
(e.g.,  between  averages)  will  in  general  de¬ 
pend  on  the  size  of  the  difference  8;  thus, 
(3  has  no  meaning  unless  associated  with  a 
particular  difference  8.  The  value  of  /8,  \ 8(8), 
associated  with  a  particular  difference  8  will 
decrease  as  8  increases.  For  a  particular 
statistical  test,  the  ability  to  detect  a  differ¬ 
ence  will  be  determined  by  three  quantities: 
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a,  /3(8),  and  n  the  sample  size.  The  comple¬ 
mentary  quantity  1 — /3  (S)  is  termed  the 
power  of  the  test  to  detect  a  difference  8  with 
a  sample  of  size  n,  when  the  test  is  carried 
out  at  the  a-level  of  significance. 

The  decision  procedure  is  a  very  logical 
one.  Suppose  we  wish  to  test  whether  two 
types  of  vacuum  tubes  have  the  same  resist¬ 
ance  in  ohms,  on  the  average.  We  take  sam¬ 
ples  of  each  type  and  measure  their  resist¬ 
ances.  If  the  sample  mean  of  one  type  of 
tube  differs  sufficiently  from  the  sample 
mean  of  the  other,  we  shall  say  that  the  two 
kinds  of  tubes  differ  in  their  average  resist¬ 
ance.  Otherwise,  we  shall  say  we  failed  to 
find  a  difference.  How  large  must  the  differ¬ 
ence  be  in  order  that  we  may  conclude  that 
the  two  types  differ,  or  that  the  observed  dif¬ 
ference  is  “significant”?*  This  will  depend 
on  several  factors :  the  amount  of  variability 
in  the  tubes  of  each  type;  the  number  of 
tubes  of  each  type;  and  the  risk  we  are 
willing  to  take  of  stating  that  a  difference 
exists  when  there  really  is  none,  i.e.,  the 
risk  of  making  an  Error  of  the  First  Kind. 
We  might  proceed  as  follows:  we  would  be 
willing  to  state  that  the  true  averages  differ, 
if  a  difference  larger  than  the  observed  dif¬ 
ference  could  arise  by  chance  less  than  five 
times  in  a  hundred  when  the  true  averages 
are  in  fact  equal.  The  probability  of  an 
Error  of  the  First  Kind  is  then  a  =  .05,  or, 
as  we  commonly  say,  we  have  adopted  a  .05 
significance  level.  The  use  of  a  significance 
level  of  .05  or  .01  is  common,  and  these  levels 
are  tabulated  extensively  for  many  tests. 
There  is  nothing  unique  about  these  levels, 
however,  and  a  test  user  may  choose  any 
value  for  a  that  he  feels  is  appropriate. 

As  we  have  mentioned,  the  ability  to  detect 
a  difference  will  in  general  depend  on  the 
size  of  the  difference  8.  Let  us  denote  by 
/3(8)  the  probability  of  failing  to  detect  a 
specified  difference  8.  If  we  plot  (i (8)  ver¬ 
sus  the  difference  8,  we  have  what  we  call 
an  Operating  Characteristic  (OC)  curve. 
Actually,  we  usually  plot  /3(8)  versus  some 


*  Or  more  accurately,  statistically  significant.  A 
difference  may  be  statistically  significant  and  yet  be 
practically  unimportant. 


convenient  function  of  8.  Figures  3-1 
through  3-8  show  OC  curves  for  a  number 
of  statistical  tests  when  conducted  at  the 
a  —  .05  or  a  =  .01  significance  levels. 

An  OC  curve  depicts  the  discriminatory 
power  of  a  particular  statistical  test.  For 
specified  values  of  n  and  a,  there  is  a  unique 
OC  curve.  The  curve  is  useful  in  two  ways. 
If  we  have  specified  n  and  a,  we  can  use  the 
OC  curve  to  read  /J(8)  for  various  values  of 
8.  If  we  are  still  at  liberty  to  set  the  sample 
size  for  our  experiment,  and  have  a  particu¬ 
lar  value  of  8  in  mind,  we  can  see  what  value 
of  n  is  required  by  looking  at  the  OC  curves 
for  specified  a.  If,  for  the  a  chosen,  the 
sample  size  required  to  achieve  a  reasonably 
small  y3(8)  is  too  large,  and  if  it  really  is 
important  to  detect  a  difference  of  8  when  it 
exists,  then  a  less  conservative  (i.e.,  larger) 
value  of  a  must  be  used.  Various  uses  of  the 
OC  curves  shown  in 'Figures  3-1  through  3-8 
are  described  in  detail  in  the  appropriate 
paragraphs  of  this  Chapter. 

It  is  evident  that  for  any  /?  (8),  n  will  in¬ 
crease  as  8  decreases.  It  requires  larger  sam¬ 
ples  to  recognize  smaller  differences.  In  some 
cases,  the  experiment  as  originally  thought 
of  will  be  seen  to  require  prohibitively  large 
sample  sizes.  We  then  must  compromise  be¬ 
tween  the  sharp  discriminatory  power  we 
think  we  need,  the  cost  of  the  amount  of  test¬ 
ing  required  to  achieve  that  power,  and  the 
risk  of  claiming  a  difference  when  none 
exists.  If  the  experiment  has  already  been 
run,  and  the  sample  size  was  fixed  from  other 
considerations,  the  OC  curve  will  show  what 
chance  the  experiment  had  of  detecting  a 
particular  difference  8. 

To  use  the  OC  curves  in  this  Chapter,  we 
must  know  the  population  standard  deviation 
<r,  or  at  least  be  willing  to  choose  some  range 
for  tr.  It  is  quite  often  possible  to  assign 
some  upper  bound  to  the  variability,  even 
without  the  use  of  past  data  (see  Paragraph 
2-2.4).  After  the  experiment  has  been  run, 
a  possibly  better  estimate  of  <x  will  be  avail¬ 
able,  and  a  hindsight  look  at  the  OC  curve 
using  this  value  will  help  to  evaluate  the 
experiment. 
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We  outline  a  number  of  different  tests  in 
this  Chapter.  For  each  test,  we  give  the  pro¬ 
cedure  to  be  followed  for  a  specified  signifi¬ 
cance  level  a  and  sample  size  n.  For  most  of 
the  tests,  we  also  give  the  OC  curve  which 
enables  us  to  obtain  the  (approximate)  value 
of  (3  for  any  given  difference.  Tables  are 
provided  for  determining  n,  the  sample  size 
required  when  a,  8,  and  /8{8)  have  been 
specified.  The  tests  given  are  exact  when : 

(a)  the  observations  for  each  item  are 
taken  randomly  from  a  single  population  of 
possible  observations;  and, 

(b)  the  quality  characteristic  measured  is 
normally  distributed  within  this  population. 
Ordinarily,  the  assumption  of  normality  is 


not  crucial,  particularly  if  the  sample  size 
is  not  very  small. 

Alternate  procedures  for  most  of  the  tests 
in  this  Chapter  are  given  in  AMCP  706-113, 
Chapters  15  and  16.  Chapter  16  gives  tests 
which  require  neither  normality  assump¬ 
tions  nor  knowledge  of  the  variability  of  the 
populations;  but  this  greater  generality  is 
achieved  at  the  -price  of  somewhat  reduced 
discriminating  power  when  normality  can  be 
assumed  and  the  knowledge  about  the  vari¬ 
ability  of  the  populations,  needed  for  the 
tests  of  this  Chapter,  is  in  hand.  Chapter 
15  gives  shortcut  tests  for  small  samples 
from  normal  populations  which  involve  less 
computation  than  the  tests  of  this  Chapter 
with  negligible  loss  of  efficiency. 


3-2  COMPARING  THE  AVERAGE  OF  A  NEW  PRODUCT 
WITH  THAT  OF  A  STANDARD 


The  average  performance  of  a  standard  prod¬ 
uct  is  known  to  be  m0.  We  shall  consider  three 
different  problems: 

(a)  To  determine  whether  the  average  of  a 
new  product  differs  from  the  standard,  Para¬ 
graph  3-2.1. 

(b)  To  determine  whether  the  average  of  a 
new  product  exceeds  the  standard.  Paragraph 
3-2.2. 

(c)  To  determine  whether  the  average  of  a 
new  product  is  less  than  the  standard.  Para¬ 
graph  3-2.3. 

For  summary  of  the  procedures  appropriate 
for  each  of  these  three  problems,  see  Table  3-1. 

It  is  necessary  to  decide  which  of  the  three 
problems  is  appropriate  before  taking  the  ob¬ 
servations.  If  this  is  not  done  and  the  choice  of 
the  problem  is  influenced  by  the  observations, 
(for  example.  Paragraph  3-2.1  vs.  3-2.2),  the 
significance  level  of  the  test,  i.e.,  the  probability 
of  an  Error  of  the  First.  Kind ,  and  the  operating 
characteristics  of  the  test  may  differ  consider¬ 
ably  from  their  nominal  values. 

Ordinarily  the  variability  of  a  new  product  is 
not  known.  At  other  times  previous  experience 
may  enable  us  to  state  a  value  of  <r.  We  shall 
outline  the  solutions  of  the  three  problems 
(Paragraphs  3-2.1,  3-2.2,  and  3-2.3)  for  both 


cases,  i.e.,  where  the  variability  is  estimated 
from  the  sample,  and  where  a  is  known  from 
previous  experience. 

Symbols  to  be  used: 

m  -  average  of  new  material,  product  or 
process  (unknown). 

m0  =  average  of  standard  material,  product 
or  process  (known). 

X  =  average  of  'Sample  of  n  measurements 
on  new  product. 

s  =  standard  deviation  estimate  computed 
from  n  measurements  on  the  new  prod¬ 
uct  (used  where  a  is  unknown). 

a  =  the  known  standard  deviation  of  the 
new  product. 

Data  Sample  3-2 — Weight  of  Powder 

For  a  certain  type  of  shell,  specifications  state 
that  the  amount  of  powder  should  average  0.735 
pound.  In  order  to  determine  whether  the 
average  for  a  new  stock  meets  the  specification, 
20  shells  are  taken  at  random,  and  the  amount 
of  powder  contained  in  each  is  weighed. 

The  sample  average  X  =  .710  pound. 

The  sample  standard  deviation  estimate 
s  =  ,0504  pound.  In  illustrating  the  known-a 
case,  we  assume  a  known  to  be  equal  to  0.06 
pound. 
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TAME  3-1.  SUMMARY  OF  TECHNIQUES  FOE  COMPARING  THE  AVIRAOC  OF  A  NEW  PRODUCT  WITH  THAT  OF  A  STANDARD 
(FOR  DETAILS  AND  WORKED  EXAMPLES  SEE  PARAGRAPHS  3-2.1,  3-2.2,  AND  3-2.3) 
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•  1th  necessary  to  h*ve  widf  value  for  *  (or  two  bounding  values)  in  order  to  use  the  Operating  Characteristic  curve  Although  w  is  unknown,  in  many  situations  It  is  pomible 
to  have  some  notion,  however  loose,  about  the  magnitude  of  «  and  thereby  to  get  helpful  information  from  the  OC  curve.  Paragraph  2-2.4  give*  assistance  in  estimating  #•  from 
general  knowledge  of  the  process. 


3-2.1  TO  DETERMINE  WHETHER  THE  AVERAGE  OF  A  NEW  PRODUCT  DIFFERS  FROM  THE 
STANDARD 


3-2. 1.1  Does  th«  Average  of  the  New  Product  Differ  from  the  Standard  (a  Unknown)? 


[Two-sided  t-test] 


Procedure 

(1)  Choose  a,  the  significance  level  of  the  test. 

(2)  Look  up  for  n  —  1  degrees  of  freedom 
in  Table  A-4. 

(3)  Compute 

X  and  s  from  the  n  measurements. 

(4)  Compute 

/  s 
u  —  1 1_„/2 

V  n 

(5)  If  |  X  —  m0\  >  u,  decide  that  the  average 
of  the  new  type  differs  from  that  of  the 
standard ;  otherwise,  that  there  is  no  reason 
to  believe  that  they  differ. 

(6)  Note:  The  interval  X  ±  u  is  a  100  (1  —  «)  % 
confidence  interval  estimate  of  the  true 
average  of  the  new  type. 


Example 

(1)  Let  a  =  .05 

(2)  f  nt  for  19  degrees  of  freedom  =  2.093 

(3) 

X  =  .710  pound 
s  =  .0504  pound 
(Data  Sample  3-2) 

(4) 

(2.093)  (.0504) 

“  “  "  v/20 

=  .0236 

(5)  |  X  -  m„ |  -  !  .710  -  .735  \  =  .025.  We  con¬ 
clude  that  the  average  amount  of  powder  in 
the  new  stock  differs  from  0.735,  the  speci¬ 
fied  standard  amount. 

(6)  Note:  .710  ±  .0236  is  a  95%  confidence  in¬ 
terval  estimate  of  the  true  average  of  the 
new  stock. 
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Operating  Characteristics  of  the  Test.  Figures  3-1  and  3-2  give  the  operating  characteristic  (OC) 
curves  of  the  preceding  test  for  a  =  .05  and  a  =  .01,  respectively,  and  various  values  of  n. 

Choose: 

5  -  jwt  —  m0j, 

the  true  absolute  difference  between  the  averages  (unknown,  of  course) 

Some  value  of  a. 

(One  may  use  an  estimate  from  previous  data;  lacking  such  an  estimate,  see  Paragraph 
2-2.4.  If  the  OC  curve  is  consulted  after  the  experiment,  we  may  use  the  estimate 
from  the  experiment.) 

Compute 


a 


We  then  can  read  from  the  OC  curve  for  a  given  significance  level  a  and  sample  size  n,  a  value 
of  0(5).  The  0(5)  read  from  the  curve  is  0(6  \  a,  a,  n),  i.e.,  0(6,  given  a,  a,  n)— the  probability  of  failing 
to  detect  this  difference  when  the  given  test  is  carried  out  with  a  sample  of  size  n,  at  the  a-level  of 
significance,  and  the  population  standard  deviation  actually  is  a. 

If  we  use  too  large  a  value  for  a,  the  effect  is  to  underestimate  d,  and  consequently  to  over¬ 
estimate  0(6),  the  probability  of  not  detecting  a  difference  of  5  when  it  exists.  Conversely,  if  we 
choose  too  small  a  value  of  a,  then  we  shall  overestimate  d  and  underestimate  0(5).  The  true  value 
of  /3(a)  is  determined,  of  course,  by  the  sample  size  n  and  the  significance  level  a  employed,  and  the 
true  value  of  a. 

Selection  of  Sample  Size  n.  If  we  choose 

the  absolute  value  of  the  average  difference  that  wc  desire  to  detect 
the  significance  level  of  the  test 
the  probability  of  failing  to  detect  a  difference  5 


then  we  may  use  Table  A-8  to  obtain  a  good  approximation  to  the  required  sample  size.  If  we  take 
a  =  .01,  then  we  must  add  4  to  the  value  obtained  from  the  table.  If  we  take  a  -  .05,  then  we 
must  add  2  to  the  table  value.  (In  order  to  compute  d,  we  must  choose  a  value  for.  a.  See  Paragraph 
2-2.4  if  no  other  information  is  available.) 

As  an  example,  suppose  that  we  plan  to  take  a  =  .05,  and  want  to  have  0  =  .50  for  a  difference 
of  .024  pound;  that  is,  we  wish  to  conduct  a  test  at  a  significance  level  of  .05  that  will  have  a  50-50 
chance  of  detecting  a  difference  of  0.024  pound.  What  sample  size  should  we  require?  Suppose 
previous  experience  suggests  that  a  lies  between  .04  and  .06  pound. 

Taking  <r  =  .04,  with  6  =  | m  -  tn0l  ~  .024,  gives  d  =  0.6.  Using  Table  A-8,  with  a  =  .05, 
1  —  /3  =  .50,  we  find  the  required  sample  size  as  n  =  11  +  2  =  13.  Taking  a  -  .06,  yields  d  -  .4. 
From  the  same  table,  we  find  that  the  required  sample  size  is  25  4-  2  =  27.  To  be  safe,  we  would 
use  n  =  27.  For  a  <  .06,  with  a  significance  level  of  .05,  this  would  give  the  two-sided  t  test  at 
least  a  50%  chance  of  detecting  a  difference  of  0.024  pound. 

If,  when  planning  an  investigation  leading  to  a  two-sided  i-test,  we  overestimate  tr,  the  conse¬ 
quences  are  two-fold:  first,  we  overestimate  the  sample  size  required,  and  thus  unnecessarily  increase 
the  cost  of  the  test;  but,  by  employing  a  sample  size  that  is  larger  than  necessary,  the  actual  value 
of  0(5)  will  be  somewhat  less  than  we  intended,  which  will  be  all  to  the  good.  On  the  other  hand,  if 


6  =  |  m  —  m0 1 , 
0, 

and  compute 


<r 
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d  or  d* 


Figure  3-1.  OC  curves  for  the  tu  o-sided  t-test  (a  =  .05). 

Adapted  with  permission  from  Anna!*  of  Mathematical  Statutiee,  Vol.  17,  No.  2,  Juno  1946,  pp.  178-197,  from  article  entitled  “Operating 
Characteristics  for  the  Common  Statistical  Test*  of  Significance'*  by  C.  D.  Ferris,  F.  E.  Grubbs,  ana  C.  L.  Weaver. 


Note:  These  curves  apply  to  the  following  tests: 

(a)  Does  the  average 'm  of  a  new  product  differ  from  a  standard  m0? 

8  —  |  to  —  mo  | 

d  —  ■ L  See  Paragraph  3-2.1. 1. 

fb)  Do  the  averages  of  two  products  differ? 

8  =  |TOa  —  TO#  J 

d*  =  I -  mD  1  1  /~w7n~B~ 

<y  Vn.4  +  nB  —  1  \nA  nB ” 

where  <xA  ~  <rB  =  a  by  assumption,  and  n*  and  nB  are  the  respective  sample  sizes  from 
products  A  and  B.  See  Paragraph  3-3. 1.1. 
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Figure  8-2.  OC  curves  for  the  two-sided  t-test  (a  =  .01). 

Adnptnd  with  pormljBion  from  Engintrtnf  SiaHtlia  by  A.  B.  Bowkor  and  G.  J.  Uvbarmno,  Copyright.  1868,  Pronticn-Hall,  Ine. 


Note:  These  curves  apply  to  the  following  tests: 


(a)  Does  the  average  m  of  a  new  product  differ  from  a  standard  mo? 
6=|  m  —  m0j 

d  =  ^  See  Paragraph  3*2. 1.1. 


(b)  Do  the  averages  of  two  products  differ? 

6  -  \mA  —  mB\ 

d*  ^  \mA  -  mB\  _ 1  ,  In  A  nB 

°  VnA  +  nB  -  1  VnA  +  ns’ 


where  aA  =  <rB  =  <r  by  assumption,  and  nA  and  nB  are  the  respective  sample  sizes  from 
products  A  and  B.  See  Paragraph  3-3.1. 1. 
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we  underestimate  a,  we  shall  underestimate  the  sample  size  actually  required,  and  by  using  too 
small  a  sample  size,  0(5)  will  be  somewhat  larger  than  we  intended,  and  our  chances  of  detecting 
real  differences  when  they  exist  will  be  correspondingly  lessened. 

The  following  brief  table,  built  around  the  preceding  example,  serves  to  illustrate  these  points 
numerically  for  a  situation  where  <*  =  .05,  and  it  is  desired  to  have  0(6)  =  .50  for  5  -\m  —  wi0|  = 
.024,  and  <r  in  fact  is  equal  to  .04  though  this  is  unknown. 

Value  of  Retulting  Correiponding 

a  Aitumed  Sample  Size  _ 0  (.024) 

.08  45  .02 

.06  27  .15 

.04  (true  value)  13  .50 

.03  9  .64 

.02  5  .80 

Thus,  if  a  actually  is  .04,  playing  safe  by  taking  a  -  .06  has  more  than  doubled  the  sample  size 
actually  needed,  but  we  have  gained  a  reduction  in  0  from  .50  to  .15. 

g 

Finally,  it  should  be  noted  that,  inasmuch  as  the  test  criterion  u  =  f  i_*/2  — ^  does  not  depend  on  a, 

an  error  in  estimating  a  when  planning  a  two-sided  f-test  will  not  alter  the  level  of  significance  of  the 
test,  which  will  be  precisely  equal  to  the  value  of  a  desired,  provided  that  1 1_„/2  is  taken  equal  to  the 
100  (1  —  a/2)  percentile  of  the  t  distribution  for  n  —  1  degrees  of  freedom,  where  n  is  the  sample 
size  actually  employed. 


3-2. 1.2  Does  the  Average  of  the  New  Product  Differ  from  the  Standard  (<r  Known)? 

(Two-tided  Normal  Teit] 


Procedure 

(1)  Choose  a,  the  significance  level  of  the  test. 

(2)  Look  up  z  i_„/2  in  Table  A-2. 

(3)  Compute 

X,  the  mean  of  the  n  measurements. 

(4)  Compute 

u^Zl-'*Vn 


(5)  If  [A'  —  mo|  >  u  decide  that  the  average 
of  the  new  type  differs  from  that  of  the 
standard;  otherwise,  that  there  is  no  reason 
to  believe  that  they  differ. 

(6)  Note  that  the  interval  X  ±  u  is  a  100 
(1  —  a)  %  confidence  interval  estimate  of 
the  true  average  m  of  the'new  type. 


Example 

(1)  Let  a  =  .05 

(2)  z.m  =  1.960 

(3) 

X  =  .710  pound 
(Data  Sample  3-2) 

(4)  a  is  known  to  be  equal  to  .06  pound. 

1.96  (.06) 

U~  V20 

*•  .0263 

(5)  j  X  -  mol  =  |  .710  -  .735|  - .025.  We  con¬ 
clude  that  there  is  no  reason  to  believe  that 
the  average  amount  of  powder  in  the  new 
stock  differs  from  0.735  (the  specified 
standard  amount). 

(6)  Note  that  (.710  zb  .0263)  is  a  95%  con¬ 
fidence  interval  estimate  for  the  true 
average  m  of  the  new  stock. 
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Operating  Characteristics  of  the  Test.  Figures  3-3  and  3-4  give  the  operating  characteristics  of  the 

preceding  test  for  a  =  .05  and  «  =  .01,  respectively.  For  any  given  n  and  d  =  lw  -  ?**■!■,  the  value 

of  0(8)  =  0(8  j  a,  a,  n ),  the  probability  of  failing  to  detect  a  difference  of  absolute  size  8  =  \m  —  m«  j , 
can  be  read  off  directly. 


Selection  of  Sample  Size  n.  If  we  specify  a,  the  significance  level,  and  0,  the  probability  or  risk  we 
are  willing  to  take  of  not  detecting  a  difference  of  absolute  size  $  =  \m  —  mo|,  then  we  can  use 
Table  A-8  to  obtain  n,  the  required  sample  size.  As  an  example,  if  <j  is  known  to  be  0.04  pound, 
and  we  wish  to  have  a  50-50  chance  of  detecting  a  difference  of  0.024  pound,  then  d  —  0.6.  From 
Table  A-8,  we  find  that  the  required  sample  size  is  11. 


When  we  know  the  correct  value  of  <r,  we  can  achieve  a  desired  value  of  0(8)  with  fewer  observa¬ 
tions  by  using  the  normal  test  at  the  desired  level  of  significance  a  than  by  using  the  corresponding 
f-test.  The  saving  is  2  or  4  observations  according  as  a  =  .05  or  .01,  respectively. 


Overestimating  or  underestimating  a  when  planning  a  two-sided  normal  test  has  somewhat 
different  consequences  than  when  planning  a  two-sided  <-test.  If  we  overestimate  a  and  choose 
<r'  >  c r,  we  also  overestimate  the  sample  size  required  as  in  the  case  of  the  f-test.  In  addition,  we 


overestimate  the  correct  test  criterion  u  =  Zi_«/i  for  the  sample  size  n  actually  adopted,  with  the 

result  that  the  effective  significance  level  of  the  normal  test  is  reduced  to  which  is  related  to  a 
by  the  equation 


^1— *'/2 


The  actual  probability  of  not  detecting  a  difference  of  5,  0'(8),  is  related  to  the  intended  risk  0(8) 
by  the  equation 


Zi-/r 


Zi-fl. 


0’(8)  will  be  less  than  0(8)  when  <r'  >  c  for  all  (large)  8  for  which  0(8)  <  0.50;  0’(8)  will  be  larger 
than  0(8)  for  all  (small)  5  for  which  0'(8)  >  0.50.  For  the  particular  8  for  which  0(8)  =>  0.50,  0’(8) 
also  will  equal  0.50.  Conversely,  if  we  underestimate  <r,  then  we  not  only  underestimate  the  sample 
size  required  but  also  the  test  criterion  for  the  sample  size  actually  used,  so  that  the  actual  risk  of 
an  Error  of  the  First  Kind  a'  will  be  larger  than  a,  and  the  risk  of  an  Error  of  the  Second  Kind 
0'(8)  will  be  increased  for  large  8,  and  decreased  for  small  5. 
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The  following  calculations  serve  to  illustrate  these  points  numerically  for  situations  bordering 
on  the  conditions  assumed  in  the  preceding  sample-size  calculation: 

Intended  significance  level  a  =  0.05. 

Intended  risk  of  Error  of  the  Second  Kind  0(6 )  =  0.50  for  6  =  0.024. 


TWO-SIDED  NORMAL  TEST 


Value  of  <r 

Assumed 

Sample  Size 
Indicated 

Actual 

Significance 

Level,  a' 

Actual  Risk  of 

Error  of  Second  Kind, 

0'  (0.024) 

.08 

43  (45)* 

.00009  (.05)* 

0.50  (.02)* 

.06 

25  (27) 

.003  (.05) 

0.50  (.15) 

.  04  (true  value) 

11  (13) 

.05  (.05) 

0.50  (.50) 

.03 

7(9) 

.14  (.05) 

0.50  (.64) 

.02 

3(5) 

.33  (.05) 

0.50  (.80) 

*  Values  in  parentheses  are  for  corresponding  two-sided  (-test. 


To  obtain  a  numerical  illustration  of  the  more  general  case  where  0(6)  y*  0.50,  let  us  modify  the 
foregoing  example  by  taking  0(5 )  =  0.20,  say,  as  the  intended  risk  of  an  Error  of  the  Second  Kind 
for  5  -  0.024: 

Intended  significance  level  «  =  0.05. 

Intended  risk  of  Error  of  the  Second  Kind  0(8)  =  0.20  for  8  =  0.024. 


TWO-SIDED  NORMAL  TEST 


Value  of  a 

Assumed 

1 

Sample  Size 
Indicated 

Actual 

Significance 

Level,  a' 

Actual  Risk  of 

Error  of  Second  Kind, 

0'  (0.024) 

.08 

88  (90)* 

.00009  (.05)* 

.046 (.0004)* 

.06 

50  (52) 

.003  (.05) 

.103  (.01) 

.04  (true  value) 

22  (24) 

.05  (.05) 

.20  (.20) 

.03 

13  (15) 

.14  (.05) 

.26  (.43) 

.02 

6(8) 

.33  (.05) 

.34  (.70) 

Values  in  parentheses  are  for  corresponding  two-sided  (-test. 
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d 

Figure  S-3.  OC  curves  for  the  two-sided  normal  test  (a  =  .05). 


Adapted  with  permission  from  Annals  of  Mathematical  Statistic*.  Vol.  17,  No.  2.  Juno  1946,  pp.  J7K-J97,  from  article  entitled  "Operating 
Characteristics  for  the  Common  Statistical  Testa  of  Significance"  by  C.  D.  Kerris,  F.  E.  Grubbs,  and  C.  L.  Weaver. 

Note:  These  curves  apply  to  the  following  tests: 

(a)  Does  the  average  m  of  a  new  product  differ  from  a  standard  ma1 

&  =  |  TO  —  Wo  | 

d  -  \m  -  See  Paragraph  3-2.1. 2. 

fJ 

(b)  Do  the  averages  of  two  products  differ? 

5  =  J  —  TOw  j 

|  mA  —  Tow  j 

a  -  ■  ,  ;  v.,  and  <rH  are  known.  See  Paragraph  3-3.1. 3. 

V  a  \  a  it 
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Figure  S-k-  OC  curves  for  the  two-sided  normal  test  (a  =  .01). 

Adapted  with  permission  from  Engineering  Sialietic «  by  A.  H.  Bowker  sod  G-  J.  Liobermsn,  Copyright,  1969.  Prentice-Hsll,  Inc. 


Note:  These  curves  apply  to  the  following  tests: 

(a)  Does  the  average  m  of  a  new  product  differ  from  a  standard  m0? 
8  ~  |  m  —  wioj 

d  =  -  ™ - —  See  Paragraph  3-2. 1.2, 

<r 


(b)  Do  the  averages  of  two  products  differ? 

8  =  |  mA  -  mB  | 

d  =  ■*?"*  ;  a A  and  <rB  are  known.  See  Paragraph  3-3.1.3. 

V  a  a  +  <tb 
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3-2.2  TO  DETERMINE  WHETHER  THE  AVERAGE  OF  A  NEW  PRODUCT  EXCEEDS  THE  STANDARD 
3-2.2.1  Does  the  Average  of  the  New  Product  Exceed  the  Standard  (<r  Unknown)? 

[Onesided  /-test] 

Procedure  Example 

(1)  Choose  a,  the  significance  level  of  the  test.  (1)  Let  a  =  .05 


(2)  Look  up  h-a  for  n  —  1  degrees  of  freedom 
in  Table  A-4. 


(3) 


Compute 
X  and  s 


(4)  Compute 

u  =  t,_„ 


s 

Vn 


(5)  If  (J?  —  m0)  >  4-  u,  decide  that  the  aver¬ 
age  of  the  new  type  exceeds  that  of  the 
standard ;  otherwise,  that  there  is  no  reason 
to  believe  that  the  average  of  the  new  type 
exceeds  that  of  the  standard. 

(6)  Note  that  the  open  interval  from  (X  —  u) 
to  +  oo  is  a  one-sided  100  (1  —  a)  %  con¬ 
fidence  interval  for  the  true  mean  of  the 
new  product. 


(2)  t.n  for  19  degrees  of  freedom  =  1.729 


(3) 


(4) 


(5) 


(6) 


X  =■  .710  pound 
s  =  .0504  pound 
(Data  Sample  3.2) 


u  = 


1.729  (.0504) 

V20 

0.019 


(X  -  m9)  =  (.710  -  .735)  -  -.025.  We 
conclude  that  there  is  no  reason  to  believe 
that  the  average  of  the  new  product  ex¬ 
ceeds  the  specified  standard. 

Note  that  the  open  interval  from  .691  to 
+  oo  is  a  one-sided  95%  confidence  inter¬ 
val  for  true  average  of  the  new  product. 


Operating  Characteristics  of  the  Test.  Figures  3-5  and  3-6  give  the  operating  characteristic  (OC) 
curves  of  the  above  test  for  a  =  .05,  and  a  -■=  .01,  respectively,  and  various  values  of  n. 

Choose: 

5  =  (m  —  mo),  the  true  difference  between  averages,  (unknown,  of  course) 

Some  value  of  a.  (We  may  use  an  estimate  from  previous  data;  lacking  such  an  estimate, 
see  Paragraph  2-2.4.  If  OC  curve  is  consulted  after  the  experiment,  we 
may  use  the  estimate  from  the  experiment.) 

Compute 


We  then  can  read  from  the  OC  curve  for  a  given  significance  level  a  and  sample  size  n,  a  value 
of  0(6).  The  0(6)  read  from  the  curve  is  0(6  j  a,  a,  n),  i.e.,  0(6  given  a,  «,  n)- — the  probability  of  failing 
to  detect  this  difference  when  the  given  test  is  carried  out  with  a  sample  of  size  n,  at  the  a-level 
of  significance,  and  the  population  standard  deviation  actually  is  <r. 

If  we  use  too  large  a  value  for  a,  the  effect  is  to  underestimate  d  and  consequently  to  overestimate 
0(6),  the  probability  of  not  detecting  a  difference  of  6  when  it  exists.  Conversely,  if  we  choose  too 
small  a  value  of  a,  then  we  shall  overestimate  d  and  underestimate  0(6).  The  true  value  of  0(6)  is 
determined,  of  course,  by  the  sample  size  n  and  significance  level  «  employed,  and  the  true 
value  of  <r. 
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Figure  3-5.  OC  curves  for  the  one-sided  t-test  ( a  =  .05). 

Adapted  with  permission  from  Engineering  Statistics  by  A.  H.  Bowker  and  G.  J.  Lieberman,  Copyright,  1959,  Prentice-Hall,  Inc. 


Note:  These  curves  apply  to  the  following  tests: 

(a)  Does  the  average  m  of  a  new  product  exceed  a  standard  m0? 

5  —  m  —  Wo 

d  —  — rn-  See  Paragraph  3-2.2. 1. 

(b)  Is  the  average  m  of  a  new  product  less  than  a  standard  m0 ? 

S  =  mc  —  m 

d  =  — - —  See  Paragraph  3-2.3. 1. 

<T 

(c)  Does  the  average  of  product  A  exceed  that  of  product  B? 

S  =  mA  —  mB 

d*  _  mA _ ,  I  nA  »» 

<r  VnA  +  nu  -  1  \nA  +  nB  ’ 

where  <?A  =  <*»  =  <r  by  assumption,  and  nA  and  nB  are  the  respective  sample  sizes  from 
products  A  and  B.  See  Paragraph  3-3.2. 1. 
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Figure  S-6.  OC  curves  for  the  one-sided  t-test  (a  -  .01). 

Adapted  with  permission  from  Enginetrino  Slalittut  by  A.  H.  Bowker  and  G.  J.  Lieberman.  Copyright,  1969,  Prentice-Hall,  Inc. 


Note:  These  curves  apply  to  the  following  tests: 

(a)  Does  the  average  m  of  a  new  product  exceed  a  standard  m0? 

S  -  m  —  m0 

d  =  m  m°  See  Paragraph  3-2. 2.1, 

(b)  Is  the  average  m  of  a  new  product  less  than  a  standard  m0? 

b  —  mn  —  m 

d  —  See  Paragraph  3-2.3.1. 

(c)  Does  the  average  of  product  A  exceed  that  of  product  B? 

5  =  mA  —  mB 

d*  _  ~  mn  1  f~nA  nB 

V  VnT+ nB  -  l  \nA  +  nB’ 

where  <rA  =  <*b  =  <r  by  assumption,  and  nA  and  nB  are  the  respective  sample  sizes  from 
products  A  and  B.  See  Paragraph  3-3.2. 1. 


3-15 


AMCP  706-110 


ANALYSIS  OF  MEASUREMENT  DATA 


Selection  of  Sample  Size  n.  If  we  choose 
5  =  (m  —  Wo), 

a,  the  significance  level  of  the  test 

0,  the  probability  of  failing  to  detect  a  positive  difference  of  size  (m  —  Too) 

and  compute 


d  = 


W  -  Wo 


a 


then  we  may  use  Table  A-9  to  obtain  a  good  approximation  to  the  required  sample  size.  If  we  are 
using  a  =  .01,  then  we  must  add  3  to  the  table  value.  If  we  are  using  a  =  .05,  then  we  must  add  2  to 
the  table  value.  (In  order  to  compute  d,  we  must  choose  a  value  for  a;  see  Paragraph  2-2.4  when 
no  other  information  is  available.) 


If,  when  planning  an  investigation  leading  to  a  one-sided  2-test,  we  overestimate  a,  the  conse¬ 
quences  are  two-fold:  first,  we  overestimate  the  sample  size  required,  and  thus  unnecessarily  increase 
the  cost  of  the  test;  but,  by  employing  a  sample  size  that  is  larger  than  necessary,  the  actual  value 
of  0(5)  will  be  somewhat  less  than  we  intended,  which  will  be  all  to  the  good.  On  the  other  hand, 
if  we  underestimate  <r,  we  shall  underestimate  the  sample  size  actually  required,  and  by  using  too 
small  a  sample  size,  0(5)  will  be  somewhat  larger  than  we  intended,  and  our  chances  of  detecting 
real  differences  when  they  exist  will  be  correspondingly  lessened.  (A  numerical  example  for  the 
too-sided  2-test  is  given  in  Paragraph  3-2.1. 1.  The  one-sided  case  is  similar). 

.  .  s 

Finally,  it  should  be  noted,  that  inasmuch  as  the  test  criterion  u  =  t i_a  does  not  depend  on  a, 

an  error  in  estimating  a  when  planning  a  one-sided  2-test  does  not  alter  the  level  of  significance  of 
the  test,  which  will  be  precisely  equal  to  the  value  of  a  desired,  provided  that  2,_0  is  taken  equal  to 
the  100  (1  —  a)  percentile  of  the  t  distribution  for  n  -  1  degrees  of  freedom,  where  n  is  the  sample 
size  actually  employed. 


3-2. 2. 2  Does  the  Average  of  the  New  Product  Exceed  the  Standard  (cr  Known)? 

[One-sided  Normal  Test] 


Procedure 

(1)  Choose  a,  the  significance  level  of  the  test. 

(2)  Look  up  z j_a  in  Table  A-2. 

(3)  Compute 

X,  the  sample  mean 

(4)  Compute 

<7 

u  =  z,.„  >' 

v  n 

(5-)  If  (X  —  mo)  >  u,  decide  that  the  average 
performance  of  the  new  type  exceeds  that 
of  the  standard ;  otherwise,  that  there  is  no 
reason  to  believe  that  the  average  of  the 
new  type  exceeds  that  of  the  standard. 

(6)  Note  that  the  open  interval  from  (Ar  —  u) 
to  f  «  isa  one-sided  100  (1  —  <»)  %  con¬ 
fidence  interval  for  the  true  mean  of  the 
new  product. 


Example 

(1)  Let  a  =  .05 

(2)  z  ti  =  1.645 

(3) 

X  =  0.710  pound 
(Data  Sample  3-2) 

(4)  u  is  known  to  be  equal  to  .06  pound. 

1.645  (.06) 

“  ~  V20 

=  .022 

(5)  ( X  -  m0)  =  .710  -  .735  =  -.025,  which 
is  not  larger  than  u.  We  conclude  that  there 
is  no  reason  to  believe  that  the  average  of 
the  new  product  exceeds  that  of  the 
standard. 

(6)  Note  that  the  open  interval  from  .688  to 
+  oo  is  a  95%  one-sided  confidence  inter¬ 
val  for  the  true  mean  of  the  new  product. 


3-16 


COMPARING  AVERAGE  PERFORMANCE  AMCP  706*110 


Operating  Characteristics  of  the  Test.  Figures  3-7  and  3-8  give  the  operating  characteristics  of  the 
above  test  for  a  =  .05  and  a  =  .01,  respectively.  For  any  given  n  and  d  =  — — — ,  the  value  of 

<T 

0(6)  =  0(5 1  <r,  a,  n),  the  probability  of  failing  to  detect  a  positive  difference  6  =  (m  —  »to),  can  be 
read  off  directly. 

Selection  of  Sample  Size  n.  If  we  specify 

8  —  (m  —  m0),  the  magnitude  of  a  positive  difference  of  interest  to  us 
a,  the  significance  level  of  the  test 

0,  the  probability  of  failing  to  detect  a  positive  difference  of  size  5 

and  compute 

,  m  —  m0 

a  —  - 

<r 

then  we  may  use  Table  A-9  to  obtain  the  required  sample  size. 


When  we  know  the  correct  value  of  a,  we  can  achieve  a  desired  value  of  0(6)  with  fewer  observa¬ 
tions  by  using  the  normal  test  at  the  desired  level  of  significance  a  than  by  using  the  corresponding 
t-test.  The  saving  is  2  or  3  observations  according  as  a  =  .05  or  .01,  respectively. 


Overestimating  or  underestimating  a  when  planning  a  one-sided  normal  test  has  somewhat 
different  consequences  than  when  planning  a  one-sided  f-test.  If  we  overestimate  a  and  choose  a'  >  <r, 
we  also  overestimate  the  sample  size  required  as  in  the  case  of  the  f-test.  In  addition,  we  overestimate 


the  correct  test  criterion  u  =  Zi_*  for  the  sample  size  n  actually  adopted,  with  the  result  that  the 
effective  significance  level  of  the  normal  test  is  reduced  to  which  is  related  to  a  by  the  equation 


Z  i—' 


£l_a* 


The  actual  probability  of  not  detecting  a  difference  of  6,  0'(5),  is  related  to  the  intended  risk  0(6) 
by  the  equation 


*  i-«. 


0'($)  will  be  less  than  0(5)  when  <r'  >  a  for  all  (large)  6  for  which  0(6)  <  0.50;  0'(6)  will  be  larger  than 
0(5)  for  all  (small)  5  for  which  0'(5)  >  0.50.  For  the  particular  6  for  which  0(5)  =  0.50,  0'(5)  also  will 
equal  0.50.  Conversely,  if  we  underestimate  a,  then  we  not  only  underestimate  the  sample  size 
required  but  also  the  test  criterion  for  the  sample  size  actually  used,  so  that  the  actual  risk  of  an 
Error  of  the  First  Kind  a'  will  be  larger  than  a,  and  the  risk  of  an  Error  of  the  Second  Kind  &'(&) 
will  be  increased  for  large  5,  and  decreased  for  small  6.  (Numerical  examples  for  the  two-sided  normal 
test  are  given  in  Paragraph  3-2.1.2.  The  one-sided  case  is  similar.) 
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Figure  3-7.  OC  curves  for  the  one-sided  normal  test  ( a  ~  .05). 

Adapted  with  permission  from  Engineering  Slalialie*  by  A.  H.  Bowker  and  G.  J-  Licberman,  Copyright,  1969,  Prentice-Hall,  Inc. 

Note:  These  curves  apply  to  the  following  tests: 

(a)  Does  the  average  m  of  a  new  product  exceed  a  standard  m0? 

5  -  m  —  m0 

d  —  — - -  See  Paragraph  3-2.2. 2. 

o 

(b)  Is  the  average  m  of  a  new  product  less  than  a  standard 

5  =  Wo  —  m 

d  =  m°  -  ™  See  Paragraph  3-2.3. 2. 

O 

(c)  Does  the  average  of  product  A  exceed  that  of  product  B? 

5  =  mA  —  mB 
mA  —  mu 

d  =  ,  <rA  and  aH  are  known.  See  Paragraph  3-3.2.3. 

Vua  4-  °b 
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*1.00  -0,50  0.0  0.50  IOO  1.50  2.00  2 JSO  300 

d 

Figure  8-8.  OC  curves  for  the  one-sided  normal  test  (a  -  .01). 


Adapted  with  permission  from  Sngintrring  Staliitici  by  A.  H.  Bowkor  and  0.  J.  I.Ieberman,  Copyright,  1959,  Prenttoo-Hall,  Ine. 

Note:  These  curves  apply  to  the  following  tests: 

(a)  Does  the  average  m  of  a  new  product  exceed  a  standard  mo? 

5  =  m  —  m0 

d  =  See  Paragraph  3*2. 2. 2. 

(b)  Is  the  average  m  of  a  new  product  less  than  a  standard  ra0? 

5  —  mQ  —  m 

d  ~  m°  ™  See  Paragraph  3-2.3.2. 

<T 

(c)  Does  the  average  of  product  A  exceed  that  of  product  B? 

S  =  mA  —  mB 

mA  —  mB  .  ,  _ 

®  „  /  y— 7 — t  ,  a  a  and  <?„  are  known.  See  Paragraph  3-3.2.3. 

V  a  a  +  0e 
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3-2.3  TO  DETERMINE  WHETHER  THE  AVERAGE  OF  A  NEW  PRODUCT  IS  LESS  THAN  THE 
STANDARD 


3-2.3. 1  li  the  Average  of  the  New  Product  Lett  than  the  Standard  (<r  Unknown)? 

(One-sided  t-fost] 


Procedure 

(1)  Choose  a,  the  significance  level  of  the  test. 

(2)  Look  up  ti-a  for  n  -  1  degrees  of  freedom 
in  Table  A-4. 

(3)  Compute 

X  and  s 


(4)  Compute 

s 

U  ~  tl— a  t— 

y/n 

(5)  If  (m0  -  X )  >  u,  decide  that  the  average 
of  the  new  type  is  less  than  that  of  the 
standard ;  otherwise,  that  there  is  no  reason 
to  believe  that  the  average  of  the  new  type 
is  less  than  the  standard. 

(6)  Note  that  the  open  interval  from  -  <*  to 
(X  +  «)  is  a  one-sided  100  (1  —  «)  %  con¬ 
fidence  interval  for  the  true  mean  of  the 
new  type. 


Example 

(1)  Let  a  =  .05 

(2)  t.n  for  19  degrees  of  freedom  =  1.729 

(3)  A'  =  .710  pound 

s  =  .0504  pound 
(Data  Sample  3-2) 

(4) 

1.729  (.0504) 

U~  V20 

-  0.019 

(5)  .735  —  .710  =  .025.  We  conclude  that  the 
average  of  the  new  type  is  less  than  that  of 
the  standard. 


(6)  Note  that  the  open  interval  from  —  °°  to 
.729  is  a  one-sided  95%  confidence  interval 
for  the  true  mean  of  the  new  type. 


Operating  Characteristics  of  the  Test.  Figures  3-5  and  3-6  give  the  operating  characteristic  (OC) 
curves  of  the  above  test  for  a  =  .05,  and  <t  =  .01,  respectively,  for  various  values  of  n. 

Choose: 

5  =  (m0  —  m),  the  true  difference  between  averages  (unknown,  of  course) 

Some  value  of  <r.  (We  may  use  an  estimate  from  previous  data;  lacking  such  an  estimate, 
see  Paragraph  2-2.4.  If  OC  curve  is  consulted  after  the  experiment,  we 
may  use  the  estimate  from  the  experiment.) 

Compute 


(T 


We  then  can  read  from  the  OC  curve  for  a  given  significance  level  a  and  sample  size  n,  a  value 
of  0(5).  The  0(5)  read  from  the  curve  is  0(5  j  a,  a,  n),  i.e.,  0(5  given  a,  a,  n) — the  probability  of  failing 
to  detect  this  difference  when  the  given  test  is  carried  out  with  a  sample  of  size  n,  at  the  a-level  of 
significance,  and  the  population  standard  deviation  actually  is  <r. 

If  we  use  too  large  a  value  for  <j,  the  effect  is  to  underestimate  d  and  consequently  to  overestimate 
0(5),  the  probability  of  not  detecting  a  difference  of  5  when  it  exists.  Conversely,  if  we  choose  too 
small  a  value  of  a,  then  we  shall  overestimate  d  and  underestimate  0(5).  The  true  value  of  0(5)  is 
determined,  of  course,  .by  the  sample  size  n  and  significance  level  a  employed,  and  the  true  value 

of  <7. 
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Selection  of  Sample  Size  n.  If  we  choose 
5  =  (mB  —  m), 

a ,  the  significance  level  of  the  test 

(3,  the  probability  of  failing  to  find  a  negative  difference  of  size  (nto  —  m); 


and  compute 
^  _  ma  —  m 

c 

then  we  may  use  Table  A-9  to  obtain  a  good  approximation  to  the  required  sample  size.  If  we  are 
using  a  =  .01,  then  we  must  add  3  to  the  table  value.  If  we  are  using  a  =  .05,  then  we  must  add 
2  to  the  table  value.  (In  order  to  use  the  table,  we  must  have  a  value  for  a.  See  Paragraph  2-2.4 
if  no  other  information  is  available.) 

The  effect  of  overestimating  or  underestimating  a  is  the  same  as  when  a  one-sided  f-test  is  to  be 
used  to  detect  a  positive  difference  of  magnitude  5  =  m  —  m0.  See  Paragraph  3-2.2. 1. 


3-2.3.2  It  lh«  Average  of  the  New  Product  Lett  Than  Thai  of  Ihe  Standard  (<r  Known)? 

[One-tided  Normal  Tett] 


Procedure 

(1)  Choose  a,  the  significance  level  of  the  test.  (1) 

(2)  Look  up  Zi_a  in  Table  A-2.  (2) 

(3)  Compute  (3) 

X,  the  sample  mean 

(4)  Compute  (4) 

<7 


U  ~  Zl~“  Vn 

(5)  If  (m0  —  X)  >  k,  decide  that  the  average  (5) 
of  the  new  type  is  less  than  that  of  the 
standard ;  otherwise,  that  there  is  no  reason 

to  believe  that  the  average  of  the  new  type 
is  less  than  that  of  the  standard. 

(6)  Note  that  the  open  interval  from  —  «  to  (6) 
(A  +  u)  is  a  one-sided  100  (1  —  a)  %  con¬ 
fidence  interval  for  the  true  mean  of  the 

new  type. 


Example* 

Let  a  —  .05 
z.n  =  1.645 


X  «  0.710  pound 
(Data  Sample  3-2) 

i t  is  known  to  be  equal  to  .06  pound. 

1.645  (.06) 

V20 
=  0.022 

(mo  —  X)  —  (.735  -  .710)  =  .025,  which 
is  larger  than  u.  We  conclude  that  the 
average  of  the  new  type  is  less  than  the 
standard. 


Note  that  the  open  interval  from  —  ■»  to 
.732  is  a  one-sided  95%  confidence  interval 
for  the  true  mean  of  the  new  type. 


Operating  Characteristics  of  the  Test.  Figures  3-7  and  3-8  give  the  operating  characteristics  of 
the  test  for  a  =  .05  and  a  =  .01,  respectively.  For  any  given  n  and  d  —  —  the  value  of 

IT 

0(5)  =  a,  n),  the  probability  of  failing  to  detect  a  negative  difference  of  size  (m<]  —  m ),  can 

be  read  off  directly. 
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Selection  of  Sample  Size  n.  If  we  specify 

5  =  (mo  -  m),  the  magnitude  of  a  negative  difference  of  interest  to  us 
a,  the  significance  level  of  the  test 

0,  the  probability  of  failing  to  detect  a  negative  difference  of  size  i, 

and  compute 
,  _  m0  —  to 

then  we  may  use  Table  A-9  to  obtain  the  required  sample  size. 

The  effect  of  overestimating  or  underestimating  <r  is  the  same  as  when  the  one-sided  normal  test 
is  to  be  used  to  detect  a  positive  difference  of  magnitude  5  =  m  —  See  Paragraph  3-2.2.2. 


3-3  COMPARING  THE  AVERAGES  OF  TWO 
MATERIALS,  PRODUCTS,  OR  PROCESSES 

We  consider  two  problems: 

(a)  We  wish  to  test  whether  the  averages  of  two  materials,  products,  or  processes  differ,  and  we 
are  not  particularly  concerned  which  is  larger,  Paragraph  3-3.1. 

(b)  We  wish  to  test  whether  the  average  of  material,  product,  or  process  A  exceeds  that  of 
material,  product,  or  process  B,  Paragraph  3-3.2. 


TABLE  3-3.  SUMMARY  OF  TECHNIQUES  FOR  COMPARING  THE  AVERAGE  PERFORMANCE  OF  TWO  PRODUCTS 
(FOR  DETAILS  AND  WORKED  EXAMPLES,  SEE  PARAGRAPHS  3-3.1  AND  3-3.2) 


Wo  With  to 
Toil 

Whether 

Para¬ 

graph 

So##*'- 

once 

Knowledge  of 
Variation 

T«»t  to  bo  Made 

Operating 
Chora  doriittcB 
of  Tart 

^  Determination  of 
Sample  Size  m 

Netei 

m*  differs 
from  v\B 

3-3-1.1 

a  a  —  Om‘,  both 
unknown 

|.T,  -  £«|  >  u,  where 
„  _  f  t  ln4  +  «e 

For  a  -  .06  and 
a  -  .01  see  Figs. 
3-1  and  3-2'  and 
Par.  3-3.1. 1. 

Use  Table  A-8.  For 
a  -  .06,  add  1  to 
the  tabular  value. 
For  a  -  .01,  add  2 
to  the  tabular  valuer 

/(»e  -  1)  e.  +  (».  -  1)  «1 
„  _  ^ - n  _  2 

3-3. 1.2 

va  i*  o;  both 
unknown 

|  Xa  —  £■]  >  u,  where 

“-''n/£+S 

See  Notes. 

V  it  the  value  of  for  the  effec¬ 

tive  number  of  degree*  of  freedom 

,  _  .^AISa+AIsjH'.  2 

ttj  +  1  n*  +  1 

3-3. 1.3 

04,  re!  both 
known 

J  X,  -  ,£»|  >  «,  where 
ni,+n. 

For  or  -  .06  and 
o  -  .01,  see  Figs. 
3-3  and  3-4. 

Use  Table  A-8. 

mA  ip  greater 
than  fltj 

3- 3.2.1 

<rA  r~  c;  both 
unknown 

i 

; 

(£a  —  Xb)  >  u,  where 

u  -  Jn‘-  ±  ”• 

v  n,  n. 

For  a  -  .05  and 
a  -  .01  see  Figs.  . 
3*5  and  8-6*  and 
Par.  8-8. 2.1. 

Use  Tahle  A-9.  For 
a  -  .06,  add  1  to 
the  tabular  value. 
For  a  -  .01,  add  2 
to  the  tabular  value. 

_  Jfet  r  IK +>.  - 
V  n,  +  n.  -  2 

3-32.2 

ca  \  both 

unknown 

, 

(Xa  -  .?»)  >  «,  where 

“  ”  »'  J-  +  - 

nf 

V  is  the  value  of  l,_  for  the  effective 
number  of  degrees  of  freedom 
,  _  W/V  +  «*g/»p)’  2 

’  (d/ns)>  (A/n.)> 

+  1  -4- 1 

3-32.3 

04,  Ob',  both 

known 

(%a  -  .£,)  >  u.  where 

u  .  2l_ 

«a 

For  a  -  .05  and 
a  -  .01  eee  Figs. 
3-7  and  3-8. 

Use  Table  A-9. 

*  Although  the  common  •  la  unknown,  useful  information  may  ba  obtained  from  the  OC  curve  II  ■  value  (or  2  bounding  value*)  of  •  pr*  be  Mourned. 
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It  again  is  important  to  decide  which  problem  is  appropriate  before  making  the  observations. 
If  this  is  not  done  and  the  choice  of  the  problem  is  influenced  by  the  observations,  the  significance 
level  of  the  test,  i.e.,  the  probability  of  an  Error  of  the  First  Kind,  and  the  operating  characteristics 
of  the  test  may  differ  considerably  from  their  nominal  values.  It  is' assumed  that  the  appropriate 
problem  has  been  selected  and  that  nA  and  n«  observations  are  taken  from  products  A  and  B, 
respectively. 

Ordinarily,  we  will  not  know  aA  or  an.  In  some  cases,  it  may  be  safe  to  assume  that  <rA  is  approxi¬ 
mately  equal  to  aB*  We  give  the  solutions  for  the  two  problems  (Paragraphs  3-3.1  and  3-3.2)  for 
three  situations  with  regard  to  knowledge  of  the  variability,  and  for  the  special  case  where  the 
observations  are  paired. 

Case  1 — The  variability  in  performance  of  each  of  A  and  B  is  unknown  but  can  be  assumed  to  be 
about  the  same. 

Case  2 — The  variability  in  performance  of  each  of  A  and  B  is  unknown,  and  it  is  not  reasonable 
to  assume  that  they  both'  have  the  same  variability. 

Cose  3 — The  variability  in  performance  of  each  of  A  and  B  is  known  from  previous  experience. 
The  standard  deviations  are  crA  and  <jr,  respectively. 

Case  U — The  observations  are  paired. 


3-3.1  DO  THE  PRODUCTS  A  AND  B  DIFFER  IN  AVERAGE  PERFORMANCE? 

3-3.1. 1  (Case  1)— Variability  of  A  and  B  is  Unknown,  But  Can  Bo  Assumed  to  be  Equal. 


Data  Sample  3-3. 1.1 — Latent  Heat  of  Fusion  of  Ice 

Two  methods  were  used  in  a  study  of  the  latent  heat  of  fusion  of  ice.  Both  Method  A  (an  electrical 
method)  and  Method  B  (a  method  of  mixtures)  were  conducted  with  the  specimens  cooled  to 
— 0.72°C.  The  data  represent  the  change  in  total  heat  from  —  0.72°C  to  water  at  0°C,  in  calories 
per  gram  of  mass. 


Method  A 

79.98 

80.04 

80.02 

80.04 

80.03 

80.03 

80.04 

79.97 

80.05 

80.03 

80.02 

80.00 

80.02 


Method  B 

80.02 

79.94 
79.98 
79.97 
79.97 
80.03 

79.95 
79.97 


*  For  a  procedure  to  test  whether  aA  and  aB  differ,  see  Chapter  4. 
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[Two-sided  t-tesf] 


Procedure 

(1)  Choose  a,  the  significance  level  of  the  test. 

(2)  Look  up  £i_„/2  for  v  =  (nA  +  nB  —  2)  de¬ 
grees  of  freedom  in  Table  A-4. 


Example 

(1)  Let  a  =  .05 

(2)  nA  =  13 
nB  -  8 

v  =  19  degrees  of  freedom 
t.,n  for  19  d.f.  =  2.093 


(3)  Compute:  XA  and  si,  XB  and  sJ„,  for  the  nA  (3) 
and  tin  measurements  from  A  and  B. 


XA  =  80.02 
si  =  .000574 
Xb  =  79.98 
si  =  .000984 


(4)  Compute 


Sp 


1)  sj  +  (nB  -  1)  sj 
nA  +  nB  -  2 


(4) 


Sp 


si 


12  (.000574)  +  7  (.000984) 
19 


V  .000725 
.0269 


(5)  Compute 


U  —  t  l_a/2  Sp 


fiA  4-  nB 
nA  nB 


(5) 


u 


=  2.093  (.0269) 


=  (.05630)  (.4493) 
=  .025 


(6)  If  \XA  —  ZB|  >  u,  decide  that  A  and  B 
differ  with  regard  to  their  average  perform¬ 
ance;  otherwise,  that  there  is  no  reason  to 
believe  A  and  B  differ  with  regard  to  their 
average  performance. 

(7)  Let  mA,  mH  be  the  true  average  perform¬ 
ances  of  A  and  B  (unknown  of  course). 
It  is  worth  noting  that  the  interval 
( XA  ~  Xb)  ±  u  is  a  100  (1  —  a)  %  con¬ 
fidence  interval  estimate  of  (mA  —  mB). 


(6)  I XA  —  =  .04,  which  is  larger  than  u. 

Conclude  that  A  and  B  differ  with  regard 
to  average  performance. 


(7)  The  interval  .04  ±  .025,  i.e.,  the  interval 
from  .015  to  .065,  is  a  95%  confidence  inter¬ 
val  for  the  true  difference  between  the 
averages  of  the  methods. 


Operating  Characteristics  of  the  Test.  Figures  3-1  and  3-2  give  the  operating  characteristic  (OC)  curves 
of  the  above  test  for  a'~  .05  and  a  —  .01,  respectively,  for  various  values  of  n  =  nA  +  na  —  1. 

Choose: 

<5  =  \  mA  —  TO*  |,  the  true  absolute  difference  between  the  averages 
Some  value  of  a  ( =  <rA  =  <jh),  the  common  standard  deviation. 

(We  may  use  an  estimate  from  previous  data;  lacking  such  an  estimate,  see  Para¬ 
graph  2-2.4.  If  OC  curve  is  consulted  after  the  experiment,  we  may  use  the  estimate 
from  the  experiment.) 

Compute 

d »  _  \**a  ~  WbI  _ 1  I  nAn„ 

a  Vn.<  +  n„  -  1  \ nA  -j-  nB 


3-24 


COMPARING  AVERAGE  PERFORMANCE 


AMCP  706-110 


We  then  can  read  a  value  of  0(8)  from  the  OC  curve  for  a  given  significance  level  and  effective  sample 
size  n  =  nA  +  nB  -  1.  The  0(8)  read  from  the  curve  is  0(6  j  a,  a,  nA,  nB)  i.e.,  0(8,  given  a,  a,  nA  and  nB) 
the  probability  of  failing  to  detect  a  real  difference  between  the  two  population  mean3  of  magnitude 
8  =  ±(mA  -  mB)  when  the  test  is  carried  out  with  samples  of  sizes  nA  and  nB,  respectively,  at  the 
a-level  of  significance,  and  the  two  population  standard  deviations  actually  ore  both  equal  to  a. 


If  we  use  too  large  a  value  for  a,  the  effect  is  to  make  us  underestimate  d*  and  consequently  to 
overestimate  0(8).  Conversely,  if  we  choose  too  small  a  value  of  a,  then  we  shall  overestimate  d* 
and  underestimate  0(8).  The  true  value  of  0(8)  is  determined,  of  course,  by  the  sample  sizes  nA  and  nB 
and  significance  level  a  actually  employed,  and  the  true  value  of  u  ( =  aA  -  aB). 


Since  the  test  criterion  u  does  not  depend  on  the  value  of  a  ( =  aA  —  aB),  an  error  in  estimating  a 
will  not  alter  the  significance  level  of  the  test,  which  will  be  precisely  equal  to  the  value  of  «  desired, 
provided  that  the  value  of  2  is  taken* equal  to  the  100  (1  —  a/2)  percentile  of  the  (-distribution 
for.  nA  -f  n»  —  2  degrees  of  freedom,  where  nA  and  nB  are  the  sample  sizes  actually  employed,  and 
it  actually  is  true  that  <rA  =  aB. 


If  aA  7^  <rB,  then,  whatever  may  be  the  ratio  aA/aB,  the  effective  significance  level  a'  will  not  differ 
seriously  from  the  intended  value  a,  provided  that  nA  =  nB,  except  possibly  when  both  are  as  small 
as  two.  If,  on  the  other  hand,  unequal  sample  sizes  are  used,  and  <j\  y*  aB,  then  the  effective  level 
of  significance  a '  can  differ  considerably  from  the  intended  value  a,  as  shown  in  Figure  3-9  where 
a  -  .05. 


0=erf  /o-*  (LOGARITHMIC  SCALE) 

Figure  3-9.  Probability  of  rejection  of  hypothesis 
mA  —  mB  when  true,  plotted  against  d. 

(A)  nA  =  nB  =  10,  P(\u\)  >  2.101 ; 

(B)  nA  =  5,  nB  =  15,P(\u\)  >  2.101. 

Adapted  with  permftafon  from  Biomelrika,  Vol.  XXIX,  ParU  III 
and  IV,  February  1938,  from  article  entitled  ‘The  aljfnincance  of  the 
difference  between  two  meani  when  the  papulation  variance*  are  un¬ 
equal"  by  B.  JL.  Welch. 
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Selection  of  Sample  Size  n.  If  we  choose 

5  =  \mA  —  mB\,  the  absolute  value  of  the  average  difference  that  we  desire  to  detect 
a,  the  significance  level  of  the  test 

0,  the  probability  of  failing  to  detect  a  difference  of  absolute  size  6, 


and  compute 
,  \mA  -  mB | 

d  =  '  ’  where  °  ~  =  °b  > 

then  we  may  use  Table  A-8  to  obtain  a  good  approximation  to  the  required  sample  size 
n  (=  nA  =  nB).  If  we  take  a  =  .01,  then  we  must  add  2  to  the  value  obtained  from  the  table. 
If  we  take  a  —  .05,  then  we  must  add  1  to  the  table  value. 

In  order  to  compute  d,  we  must  choose  a  value  for  a  ( =  aA  =  c rB).  (See  Paragraph  2-2.4  if  no 
other  information  is  available.)  If  we  overestimate  a,  the  consequences  are  two-fold:  first,  we  over¬ 
estimate  the  sample  size  n  (  =  nA  =  nB )  required,  and  thus  unnecessarily  increase  the  cost  of  the 
test;  but,  by  employing  a  sample  size  that  is  larger  than  necessary,  the  actual  value  of  0(6)  will  be 
somewhat  less  than  we  intended,  which  will  be  all  to  the  good.  On  the  other  hand,  if  we  under¬ 
estimate  a,  we  shall  underestimate  the  sample  size  actually  required,  and  by  using  too  small  a 
sample  size,  0(6)  will  be  somewhat  larger  than  we  intended,  and  our  chances  of  detecting  real 
differences  when  they  exist  will  be  correspondingly  lessened.  These  effects  of  overestimating  or 
underestimating  a  (=  aA  =*  v8)  will  be  similar  in  magnitude  to  those  considered  and  illustrated  in 
Paragraph  3-2. 1.1  for  the  case  of  comparing  the  mean  to  of  a  new  material,  product,  or  process, 
with  a  standard  value  m0. 

As  explained  in  the  preceding  discussion  of  the  operating  characteristics  of  the  test,  an  error  in 
estimating  a  ( -  <rA  =  <t»)  will  have  no  effect  on  the  significance  level  of  the  test,  provided  that  the 
value  of  <i  .a/ 2  is  taken  equal  to  the  100  (1  -  a/2)  percentile  of  the  f-distribution  for  nA  -f  nB  —  2 
degrees  of  freedom,  where  nA  and  nH  are  the  sample  sizes  actually  employed;  and  if  aA  <rB,  the 
effect  will  not  be  serious  provided  that  the  sample  sizes  are  taken  equal. 


3-3. 1.2  (Cate  2) — Variability  of  A  and  8  if  Unknown,  Cannot  Be  Assumed  Equal. 

Data  Sample  3-3. 1.2 — Compressive  Strength  of  Concrete 


Two  investigators  using  somewhat  different  techniques  obtained  specimen  cores  to  determine  the 
compressive  strength  of  the  concrete  in  a  poured  slab.  The  following  results  in  psi  were  reported: 


A 

B 

3128 

1939 

3219 

1697 

3244 

3030 

3073 

2424 

2020 

2909 

1815 

2020 

2310 
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Procedure*  Example 

(1)  Choose  <x,  the  significance  level  of  the  test.  (1)  Let  a  =  .05 
(Actually,  the  procedure  outlined  will  give 
a  significance  level  of  only  approximately 
a). 


(2)  Compute:  XA  and  si,  XB  and  si,  for  the  nA  (2) 
and  Tin  measurements  from  A  and  B. 


(3)  Compute:  (3) 


and 


the  estimated  variances  of  XA  and  XB, 
respectively. 


XA  =  3166.0 
si  =  6328.67 
nA  =  4 
XB  =  2240.4 
si  =  221,661.3 
nfl  =  9 


Va  - 


6328.67 

4 


=  1582.17 


221,661.3 

9 


=  24629.03 


(4) 


Compute  the  “effective  number  of  degrees 
of  freedom" 

,  (Va  +  Vi,)2  9 

_  Va  ,  Vi 

nA  +  1  Ms  -f-  1 


(4) 


(26211.20)’ 

J  “  500652.4  +  60658911.9 
_  687027005  „ 

61159564 
=  11.233  -  2 
=  9.233 


(5)  Look  up  ti-ef  t  for  /'  degrees  of  freedom  in  (5)  /'  =  9 

Table  A-4,  where/'  is  the  integer  nearest  to  =  2.262 

f;  denote  this  value  by 


(6)  Compute 

M  =  tl-a/i  \^VA  +  Vg 


(6) 


u  =  2.262  V26211.20 
=  2.262  (161.9) 

=  366.2 


(7)  If  \XA  —  >  u,  decide  that  A  and  B 

differ  with  regard  to  their  average  perform¬ 
ance;  otherwise,  decide  that  there  is  no 
reason  to  believe  A  and  B  differ  in  average 
performance. 


(7)  |  XA  -  X»  |  =  925.6,  which  is  larger  than 
u.  Conclude  that  A  and  B  differ  with 
regard  to  average  performance. 


*  See  footnote  on  page  3-28. 
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Procedure* 

(8)  If  mA,  mB  are  the  true  average  perform¬ 
ances  of  A  and  B  (unknown  of  course), 
then  it  is  worth  noting  that  the  interval 
(XA  —  XB )  ±  »  is  approximately  a 
100  (1  —  a)  %  confidence  interval  estimate 
of  mA  —  mB. 


Example 

(8)  The  interval  925.6  ±  366.2,  i.e.,  the  inter¬ 
val  from  559.4  psi  to  1291.8  psi  is  a  95% 
confidence  interval  for  the  true  difference 
between  the  averages  of  the  two  methods. 


Discussion. 

To  gain  some  understanding  of  the  nature,  properties,  and  limitations  of  this  approximate 
procedure,  note  first  that  VA  and  V B  are  unbiased  estimates  of  the  true  variances  <PA/nA  and 
of  the  means  XA  and  XB,  respectively.  Consequently,  V A  +  V  B  is  an  unbiased  estimate  of  the  true 
variance  of  the  difference  XA  —  XB,  provided  only  that  XA  and  XB  are  the  means  of  independent 
random  samples  of  nA  and  nB  observations  from  populations  A  and  B,  respectively.  Note  next  that 
the  effective  number  of  degrees  of  freedom  /,  defined  by  the  expression  in  step  (4),  also  can  be 
expressed  in  the  form 

__1 _ c5__  (1  -  c)» 

/  +  2  fA  -h  2  fB-\-  2 

where 

} a  —  nA  —  1  and  fB  =  ns  -  1 

are  the  degrees  of  freedom  associated  with  the  variance  estimates  V A  and  V B,  respectively,  and, 

c  =  VT+Vs  and  1  ~ c  =  v7+~vl 

are  the  fractions  of  the  estimated  variance  of  the  difference  XA  —  XB  that  are  associated  with 
XA  and  XB,  respectively.  From  this  expression  for  /,  it  is  evident  that  /  can  never  be  less  than  the 
smaller  of//  ( =  nA  —  1)  and  fB  ( =  nB  —  1),  and /cannot  be  larger  than 


(fA  +  2)  +  (/a  +  2)  —  2  ==  nA  nB. 


When  V A  is  so  large  in  comparison  to  Fa  that  V B  is  negligible,  then  c  ~  1  and  /  ~  fA,  which  is 
intuitively  reasonable — the  fB  degrees  of  freedom  upon  which  VH  is  based  are  not  making  a  useful 
contribution  to  the  estimate  of  the  variance  of  the  difference  XA  —  XB ■  Similarly,  when  V B 
dominates  the  situation,  then  c  ~  0  and  /  ~  fB.  In  intermediate  situations  where  neither  VA  nor  V B 
can  be  neglected,  both  the  fA  and  the  fB  degrees  of  freedom  make  useful  contributions,  and  the 
effective  number  of  degrees  of  freedom  /  expresses  the  sum  of  their  joint  contributions.  Thus,  in 
our  illustrative  example,//  =  3  and/fi  =  8,  but /  =  9\  Both  samples  make  their  maximum  con¬ 
tributions,  that  is,  /  achieves  its  maximum  of  nA  +  nB,  only  when  VA/VB  =  (nA  +  1  )/(nB  +  1), 
i.e.,  when  s^/4  =  nA  (nA  +  1  )/n»  (n«  +  1), 


*  The  test  procedure  given  here  is  an  approximation,  i.e.,  the  stated  significance  level  is  only  approximately  achieved. 
The  approximation  is  good  provided  nA  and  nB  are  not  too  small.  A  more  accurate  procedure  is  given  in  Biomeirika 
Tables  for  Statisticians ,<*>  which  (in  the  notation  of  the  present  procedure)  provides  10%  and  2%  significance  levels  of 
it)!  =  |(Xx  -  XB)  -  (mA  -  tob)|/VVx  +  V*  for  nA  >  6,  n8  >  6,  and  0  <  V A/(V A  +  VB)  <  1.  5%  and  1%  sig¬ 

nificance  levels  of  1  r  I  for  >  8  and  nB  >  8  and  the  same  range  of  V A/(V A  +  Vs)  are  given  by  Trickett,  Welch,  and 
James.'2'  (When  using  either  of  the  tables  (1)  or  (2),  it  should  be  noticed  that  our  “a"  corresponds  to  their  "2a”.) 

The  appropriate  modification  when  the  value  of  the  ratio  of  the  variances  9  =  is  known,  but  not  their  respective 

values,  is  indicated  at  the  end  of  the  Discussion  that  follows  this  procedure. 
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When  samples  of  equal  size  ( nA  =  nB  =  n)  are  involved,  the  present  approximate  procedure  for 
Case  2  {uA  and  uB  both  unknown  and  presumably  unequal )  in  Paragraph  3-3.1.2,  and  the  exact 
procedure  for  Case  1  (aA  and  crB  presumably  equal,  but  their  common  value  unknown)  given  in 
Paragraph  3-3.1. 1  are  the  same  in  all  respects  except  for  the  value  of  to  be  used.  In  the  exact 
procedure  for  Case  1,  the  value  of  to  be  used  when  nA  -  nB  =  n  is  the  100  (1  —  a/2)  per¬ 
centile  of  the  /-distribution  for  v  =  2  (n  —  1)  degrees  of  freedom,  and  is  completely  determined  by 
the  choice  of  significance  level  a  and  the  common  sample  size  n.  In  contrast,  the  value  of  ti-*/ j  to 
be  used  in  the  approximate  procedure  for  Case  2  when  nA  —  nB  =  n  is  the  100  (1  —  a/2)  percentile 
for  the  integral  number  of  degrees  of  freedom  /'  nearest  to  the  effective  degrees  of  freedom 


/  =  (n  +  1) 


L4  +  &)* 

S*A  +  S% 


-2, 


and  thus  depends  not  only  on  the  choice  of  significance  level  a  and  common  sample  size  n,  but  also 
on  the  ratio  sA/sB  of  the  sample  estimates  of  <j\  and  a'B.  Furthermore,  since /  can  vary  from  (n  —  1) 
to  2n,  and  equals  2 n  only  when  sA  =  s’0,  it  is  clear  that  the  two  procedures  may  lead  to  different 
results  when  <rA  ~  aB.  Consequently,  when  samples  of  equal  size  (nA  =  nB  —  n)  are  involved,  the 
procedure  for  Case  1  of  Paragraph  3-8. 1.1  should  be  used  even  when  it  cannot  be  assumed  that  <xA  ~  <jb. 
If  in  fact  uA  =  <tb,  then  the  effective  significance  level  a'  will  be  identically  equal  to  the  intended 
significance  level  a,  and  the  test  will  have  maximum  sensitivity  with  respect  to  any  real  difference 
between  the  population  means  mA  and  mB.  If,  on  the  other  hand,  crA  ^  <?„,  then  the  effective 
significance  level  a  will  differ  from  the  significance  level  a  intended,  but  only  slightly,  as  shown  by 
curve  (A)  in  Figure  3-9;  and  the  test  will  tend  to  have  greater  sensitivity  with  respect  to  any  real 
difference  between  mA  and  mB  than  would  be  the  case  if  the  procedure  of  the  present  section 
were  used. 


In  contrast,  when  the  samples  are  of  unequal  size  (nA  ^  nB),  the  procedure  of  the  present  section 
should  always  be  used  unless  it  is  known  for  certain  that  <r ,  =  aB.  Otherwise,  the  effective  significance 
level  a'  may  differ  considerably  from  the  significance  level  a  intended,  even  when  aA  ~  <jB  as  shown 
by  curve  (B)  in  Figure  3-9. 

When  the  smaller  sample  comes  from  the  more  variable  population,  the  effective  number  of 
degrees  of  freedom  /  to  be  used  with  the  procedure  of  the  present  section  is  likely  to  be  much  smaller 
than  nA  +  nB  -  2,  the  degrees  of  freedom  to  be  used  with  the  procedure  of  Paragraph  3-3.1. 1. 
Nevertheless,  the  small  advantage  of  greater  sensitivity  to  real  differences  between  mA  and  mB  that 
the  procedure  of  Paragraph  3-3. 1.1  provides  when  <rA  -  <ra  is  rapidly  offset,  as  the  inequality  of 
<rA  and  <rB  increases,  by  the  much  firmer  control  of  the  effective  significance  level  by  the  procedure 
of  the  present  section,  except  when  /  is  very  small  (say  <  6). 

Finally,  it  should  be  remarked  that  the  effective  number  of  degrees  of  freedom  appropriate  to  the 
procedure  of  the  present  section  is  given  more  accurately  by 

f*  =  («a  +  rg)*  =  (; nBe  d-  nAy 

_ .  .  ,  Vb_  _n n\ 
nA  —  1  "r  nB  -  1  nA  -  1  _t’  n\ 

where 

<7^,  ,  (A 

vA  =  —  and  vs  =  — 

nA  nB 

are  the  true  variances  of  XA  and  fCB,  respectively,  and  0  — 
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It  easily  is  shown  that  /*  never  is  less  than  the  smaller  of  —  1  and  nB  —  1,  and  never  exceeds 
nA  +  nB  —  2.  If  we  know  the  values  of  and  a*B,  then  we  could  evaluate  /*;  but  under  these 
circumstances  we  should  use  the  procedure  of  Paragraph  3-3. 1.3,  not  the  present  approximate 
procedure.  If  we  do  not  know  the  values  of  ^  and  crj,,  but  do  know  their  ratio  6,  then  the  exact 
procedure  (Case  3)  of  Paragraph  3-3.1. 3  cannot  be  applied,  but/*  can  be  evaluated.  Under  these 
circumstances,  the  approximate  procedure  of  the  present  section  should  be  followed,  with /replaced 
by  /*.  When  we  do  not  know  the  values  of  and  <rl,  nor  even  their  ratio  6,  then  we  must  rely  on 
the  best  available  sample  estimate  of  /*;  namely,  /  defined  in  Step  (4)  of  the  present  procedure. 


3-3.1 .3  (Case  3) — Variability  in  Performance  of  Each  of  A  and  B  is  Known  from  Previous  Experience, 
and  the  Standard  Deviations  are  uA  and  <jb,  respectively. 


Data  Sample  3-3. 1.3 — Latent  Heat  of  Fusion  of  Ice 

The  observational  data  are  those  of  Data  Sample  3-3. 1.1  and,  in  addition,  it  now  is  assumed  to 
be  known  that  aA  =  0.024  and  <rn  =  0.033  . 

[Two-sided  Normal  Test] 


Procedure 

(1)  Choose  a,  the  level  of  significance  of  the 
test. 

(2)  Look  up  Zi_a/2  in  Table  A-2. 

(3)  Compute:  XA  and  XB,  the  means  of  the  nA 
and  nB  measurements  from  A  and  B. 


(4)  Compute 


u~Zl-^yjrA  + 


£* 

n„ 


(5)  If  |  XA  —  A'bJ  >  u,  decide  that  A  and  B 
differ  with  regard  to  their  average  perform¬ 
ance;  otherwise,  decide  that  there  is  no 
reason  to  believe  that  A  and  B  differ  in 
average  performance. 

(6)  Let  mA,  ms  be  the  true  average  perform¬ 
ances  of  A  and  B  (unknown  of  course). 
It  is  worth  noting  that  the  interval 
(XA  —  XB)  ±  tt  is  a  100  (l  —  a)  %  con¬ 
fidence  interval  estimate  of  (mA  —  mB). 


Example 

(1)  Let  a  =  .05 


(2)  z.975  =  1.960 

(3)  XA  =  80.02 

=  0.000576 
nA  =  13 

XB  =  79.98 

<j\  =  0.001089 

nB  -  8 

(4) 

..  •.  /0t000576  ,'  0.001089 

u  -  1.960  ^  ^  y  g 

=  1.960  (.0134) 

=  0.026 

(5)  —  XB  \  —  .04,  which  is  larger  than  u. 
Conclude  that  methods  A  and  B  differ  with 
regard  to  their  averages. 


(6)  The  interval  .04  ±  .026  i.e.,  the  interval 
from  .014  to  .066  is  a  95%  confidence  inter¬ 
val  for  the  true  difference  between  the 
averages  of  the  methods. 
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Operating  Characteristics  of  the  Test.  Figures  3-3  and  3-4  give  the  operating  characteristic  (OC) 
curves  of  the  above  test  for  a  =  .05  and  a  =  .01,  respectively,  for  various  values  of  n. 

If  nA  -  nH  =  n,  and  (mA  —  mB)  is  the  true  difference  between  the  two  averages,  then  putting 

,  jmA  -  mB  | 

i  -  ■7*  +  %' 

we  can  read  0,  the  probability  of  failing  to  detect  a  difference  of  size  ±  (mA  —  mB). 

If  nA  —  cnB,  we  can  put  d  -  -^^=====j,  and,  using  n  -  nA,  we  can  read  /3,  the  probability  of 

"v  <y.4  i  CP# 

failing  to  detect  a  difference  of  size  ±  (mA  —  mB). 

Selection  of  Sample  She.  We  choose 
a,  the  significance  level  of  the  test 

0,  the  probability  of  failing  to  detect  a  difference  of  size  (mA  —  mB). 

If  we  wish  nA  =  n!s  —  n,  we  compute 

,  |  mA  -  mB  | 

d~v^+* 

and  we  may  use  Table  A-8  directly  to  obtain  the  required  sample  size  n. 

If  we  wish  to  have  nA  and  nB  such  that  nA  =  cnB,  then  we  may  compute 

d  _  1  m*  ~  m»\ 

<ta  +  Cai 

and  use  Table  A-8  to  obtain  n  -  nA. 


3-3. K4  (Cas«  4) — The  Observation*  are  Paired. 

Often,  an  experiment  is,  or  can  be,  designed  so  that  the  observations  are  taken  in  pairs.  The  two 
whits  of  a  pair  are  chosen  in  advance  so  as  to  be  as  nearly  alike  as  possible  in  all  respects  other  than 
the  characteristic  to  be  measured,  and  then  one  member  of  each  pair  is  assigned  at  random  to  treat¬ 
ment  A,  and  the  other  to  treatment  B.  For  instance,  the  experimenter  may  wish  to  compare  the 
effects  of  two  different  treatments  on  a  particular  type  of  device,  material,  or  process.  The  word 
"treatments”  here  is  to  be  understood  in  a  broad  sense:  the  two  "treatments”  may  be  different 
operators;  different  environmental  conditions  to  which  a  material  may  be  exposed,  or  merely  two 
different  methods  of  measuring  one  of  its  properties;  two  different  laboratories  in  an  interlaboratory 
test  of  a  particular  process  of  measurement  or  manufacture.  Since  the  comparison  of  the  two  treat¬ 
ments  is  made  within  pairs,  two  advantages  result  from  such  pairing.  First,  the  effect  of  extraneous 
variation  is  reduced  and  there  is  consequent  increase  in  the  precision  of  the  comparison,  and  in  its 
sensitivity  to  real  differences  between  the  treatments  with  respect  to  the  measured  characteristic. 
Second,  the  test  may  be  carried  out  under  a  wide  range  of  conditions  representative  of  actual  use 
without  sacrifice  of  sensitivity  and  precision,  thereby  assuring  wider  applicability  of  any  conclusions 
reached. 
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Data  Sample  3-3. 1.4— Capacity  of  Batteries 


The  data  below  are  measurements  of  the  capacity  (in  ampere  hours)  of  paired  batteries,  one  from 
each  of  two  different  manufacturers: 


X„  =  XA  -  X, 


A 

146 

141 

135 

142 
140 

143 
138 
137 
142 

136 


B 

141 

143 

139 

139 

140 

141 
138 
140 

142 
138 


5 

-2 

-4 

3 

0 

2 

0 

-3 

0 

-2 


Procedure 

(1)  Choose  a,  the  significance  level  of  the  test. 

(2)  Compute:  Xd  and  sd  for  the  n  differences, 
Xd.  (Each  Xd  represents  an  observation  on 
A  minus  the  paired  observation  on  B). 

(3)  Look  up  for  n  —  1  degrees  of  freedom 
in  Table  A-4. 

(4)  Compute 

M  Vn 


(5)  If  |  Xd\  >  u,  decide  that  the  averages  dif¬ 
fer;  otherwise,  that  there  is  no  reason  to 
believe  they  differ. 

(6)  Note:  The  interval  Xd  ±  u  is  a 
100  (1  —  a)  %  confidence  interval  estimate 
of  the  average  difference  (A  minus  B). 


Example 

(1)  Let  a  =  .05 

(2)  Xd  «  -0.1 

srf  =  2.807 


(3)  t.,7i  (9  d.f.)  =  2.262 


(4) 

“  - 2262  (Hi) 

-  2.008 

(5)  |  Xd  \  =  0.1,  which  is  less  than  u.  Conclude 
that  batteries  of  the  two  manufacturers  do 
not  differ  in  average  capacity. 

(6)  The  interval  -0.1  ±  2.0,  i.e.,  the  interval 
—  2.1  to  -f  1.9  is  a  95%  confidence  interval 
estimate  of  the  average  difference  in 
capacity  between  the  batteries  of  the  two 
manufacturers. 


Operating  Characteristics  of  the  Test.  Figures  3-1  and  3-2  give  the  operating  characteristic  (OC) 
curves  of  the  above  test  for  a  =  .05  and  a  =  .01,  respectively,  for  various  values  of  n,  the  number 
of  pairs  involved. 
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Choose: 

5  =  \mA  -  mB\,  the  true  absolute  difference  between  the  averages  (unknown, 
of  course) 

Some  value  of  a  ( =  a*),  the  true  standard  deviation  of  a  signed  difference  Xd. 

(We  may  use  an  estimate  from  previous  data.  If  OC  curve  is  consulted  after  the 
experiment,  we  may  use  the  estimate  from  the  experiment.) 

Compute 


(T 


We  then  can  read  from  the  OC  curve  for  a  given  significance  level  a  and  sample  size  n,  a  value  of 
of  (8(5).  The  fi(S)  read  from  the  curve  is  8(5  \  a,  a,  n),  i.e.,  8(6,  given  a,  a,  n ) — the  probability  of 
failing  to  detect  a  difference  of  ±.  (mA  —,mB)  when  it  exists,  if  the  given  test  is  carried  out  with  n 
pairs,  at  the  a-level  of  significance,  and  the  standard  deviation  of  signed  differences  Xd  actually  is  <r. 


If  we  use  too  large  a  value  for  <j,  the  effect  is  to  underestimate  d,  and  consequently  to  overestimate 
8(5),  the  probability  of  not  detecting  a  difference  of  5  when  it  exists.  Conversely,  if  we  choose  too 
small  a  value  of  a,  then  we  shall  overestimate  d  and  underestimate  8(6)-  The  true  value  of  8(5)  is 
determined,  of  course,  by  the  sample  size  n  and  the  significance  level  a  employed,  and  the  true 
value  of  a  (  =  <u). 


Selection  of  Number  of  Pairs  n  required.  If  we  choose 

5  =  ( mA  —  mB  |,  the  absolute  value  of  the  average  difference  that  we  desire  to  detect 
a,  the  significance  level  of  the  test 
8,  the  probability  of  failing  to  detect  a  difference  of  5 

and  compute 
d  =  |  mA  - 
a 

where  a  is  the  standard  deviation  of  the  population  of  signed  differences  Xd  for  the  type  of  pairs 
concerned,  then  we  may  use  Table  A-8  to  obtain  a  good  approximation  to  the  required  number  of 
pairs  n.  If  we  take  a  =  .01,  then  we  must  add  4  to  the  value  obtained  from  the  table.  If  we  take 
a  =  .05,  then  we  must  add  2  to  the  table  value.  In  order  to  compute  d,  we  must  choose  a  value  for  <r. 

If,  when  planning  the  tost,  we  overestimate  a,  the  consequences  are  two-fold:  first,  we  over¬ 
estimate  the  number  of  pairs  required,  and  thus  unnecessarily  increase  the  cost  of  the  test;  but,  by 
employing  a  sample  size  that  is  larger  than  necessary,  the  actual  value  of  8(6)  will  be  somewhat 
less  than  we  intended,  which  will  be  all  to  the  good.  On  the  other  hand,  if  we  underestimate  a, 
we  shall  underestimate  the  number  of  pairs  actually  required,  and  by  using  too  small  a  sample 
size,  8(5)  will  be  somewhat  larger  than  we  intended,  and  our  chances  of  detecting  real  differences 
when  they  exist  will  be  correspondingly  lessened. 


$d 

Finally,  it  should  be  noted,  that  inasmuch  as  the  test  criterion  u  -  does  not  depend  on  <r, 


an  error  in  estimating  <j  when  planning  the  test  will  not  alter  the  level  of  significance,  which  will  be 
precisely  equal  to  the  value  of  a  desired,  provided  that  is  taken  equal  to  the  100  (1  —  a/2) 
percentile  of  the  ^-distribution  for  n  —  1  degrees  of  freedom,  where  n  is  the  number  of  pairs  actually 
employed. 
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3-3.2  DOES  THE  AVERAGE  OF  PRODUCT  A  EXCEED  THE  AVERAGE  OF  PRODUCT  B? 


3-3.2. 1  (Cat*  1 )— Variability  of  A  and  B  it  Unknown,  but  can  be  Attorned  to  be  Equal. 

Data  Sample  3-3.2.1— Surface  Hardnett  of  Steel  Platet 

A  study  was  made  of  the  effect  of  two  grinding  conditions  on  the  surface  hardness  of  steel  plates 
used  for  intaglio  printing.  Condition  A  represents  surfaces  "as  ground”  and  Condition  B  represents 
surfaces  after  light  polishing  with  emery  paper.  The  observations  are  hardness  indentation  numbers. 


Condition  A 

187 

157 

152 

164 

159 

164 

172 


Condition  B 

157 
152 
148 

158 
161 


[One-tided  t-tett] 


Procedure 

(1)  Choose  a,  the  significance  level  of  the  test. 

(2)  Look  up  for  v  =  nA  +  nB  —  2  degrees 
of  freedom  in  Table  A-4. 


(3)  Compute:  Ax  and  Sx,  XB  and  Ss,  from  the 
nA  and  nB  measurements  from  products  A 
and  B,  respectively. 


(4) 


Compute 
s,  -  ^ 


-  1)  s\  +  ( nB  -  1)  4 
nA  +  nB  -  2 


(5)  Compute 

u  =  ( i_ 


sr 


4 


nA  +  nB 
nA  nB 


(6)  If  (Ax  —  XB)  >  u,  decide  that  the  average 
of  A  exceeds  the  average  of  B;  otherwise, 
decide  there  is  no  reason  to  believe  that  the 
average  of  A  exceeds  the  average  of  B. 

(7)  Let  mA  and  mB  be  the  true  averages  of  A 
and  B.  Note  that  the  interval  from 
((Ax  -  XB)  -u)to»isal-a  one¬ 
sided  confidence  interval  estimate  of  the 
true  difference  ( mA  —  mB). 


Example 

(1)  Let  a  =  .05 

(2)  nA  =  7 

nB  =  5 

v  =  10 

t. n  for  10  d.f.  -  1.812 

(3)  Ax  =  165 

=  134 
XB  =  155.2 
sj,  -  26.7 

(4) 

„  /6  (134) +4(26.7) 

Si  "  V  “10 

=  \/91.08 
=  9.544 

(5) 

u  =  (1.812)  (9.544)  ^|| 

=  17.294  (.5855) 

=  10.1 

(6)  (Ax  —  XB)  =  9.8,  which  is  not  larger  than 

u.  There  is  no  reason  to  believe  that  the 
average  hardness  for  Condition  A  exceeds 
the  average  hardness  for  Condition  B. 

(7)  (Ax  -  XB)  -  u  =  9.8  -  10.1  =  -0.3. 
The  interval  from  -  0.3  to  «  is  a  95%  one¬ 
sided  confidence  interval  estimate  of  the 
true  difference  between  averages. 
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Operating  Characteristics  of  the  Test.  Figures  3-5  and  3-6  give  the  operating  characteristic  (OC)  curves 
of  the  above  test  for  a  =  .05  and  a  =  .01,  respectively,  for  various  values  of  n  =  nA  +  nB  —  1. 

Choose: 

8  -  (mA  —  ««),  the  true  difference  between  the  averages 
Some  value  of  a  (--  a  a  ~  <r*),  the  common  standard  deviation 
(We  may  use  an  estimate  from  previous  data;  lacking 
such  an  estimate,  see  Paragraph  2-2.4.  If  OC  curve  is 
consulted  after  the  experiment,  we  may  use  the  estimate 
from  the  experiment). 

Compute 

(mA  -  mB)  _  1_  jn^UB 

<r  VnA  +  nB  -  i\nA  +  nB' 

We  then  can  read  a  value  of  0(5)  from  the  OC  curve  for  a  given  significance  level  and  effective 
sample  size  n.  The  0(6)  read  from  the  curve  is  0(5 \a,  a,  nA,  nB)  i.e.,  0(6,  given  o,  a,  nA,  and  nB)  the 
probability  of  failing  to  detect  a  real  difference  between  the  two  population  means  of  magnitude 
6  =  +  ( mA  —  m„)  when  the  test  is  carried  out  with  samples  of  sizes  nA  and  n„,  respectively,  at 
the  a-level  of  significance,  and  the  two  population  standard  deviations  actually  are  both  equal  to  o. 

If  we  use  too  large  a  value  for  o,  the  effect  is  to  make  us  underestimate  d*,  and  consequently  to 
overestimate  0(6).  Conversely,  if  we  choose  too  small  a  value  of  <x,  then  we  shall  overestimate  d* 
and  underestimate  0(8).  The  true  value  of  0(5)  is  determined,  of  course,  by  the  sample  sizes 
( nA  and  nn )  and  significance  level  a  actually  employed,  and  the  true  value  of  a  (=  aA  =  on). 

Since  the  test  criterion  u  does  not  depend  on  the  value  of  c  ( =  oA  =  oB),  an  error  in  estimating  a 
will  not  alter  the  significance  level  of  the  test,  which  will  be  precisely  equal  to  the  value  of  a  desired, 
provided  that  the  value  of  fi_„  is  taken  equal  to  the  100  (1  —  a)  percentile  of  the  (-distribution  for 
nA  +  nB  -  2  degrees  of  freedom,  where  nA  and  nB  are  the  sample  sizes  actually  employed,  and  it 
actually  is  true  that  <jA  -  oB. 

If  a A  ^  oB,  then,  whatever  may  be  the  ratio  oA/oB,  the  effective  significance  level  «'  will  not 
differ  seriously  from  the  intended  value  a,  provided  that  nA  =  nB,  except  possibly  when  both  are  as 
small  as  two.  If,  on  the  other  hand,  unequal  sample  sizes  are  used,  and  <rA  ^  aB,  then  the  effective 
level  of  significance  a'  can  differ  considerably  from  the  intended  value  a,  as  shown  in  Figure  3-9. 

Selection  of  Sample  Size  n.  If  we  choose 

5  =  (to.<  —  m„),  the  value  of  the  average  difference  that  we  desire  to  detect 
<t,  the  significance  level  of  the  test 

0,  the  probability  of  failing  to  detect  a  difference  of  size  8 

and  compute 

(mA  -  m„) 

a  =  \y/2 a*  ’  w')ere  a  ~  °A  ~  <r"  ’ 

then  we  may  use  Table  A-9  to  obtain  a  good  approximation  to  the  required  sample  size 
n(—  nA  =  nB).  If  we  take  «  =  .01,  then  we  must  add  2  to  the  table  value.  If  we  take  a  =  .05, 
then  we  must  add  1  to  the  table  value. 

In  order  to  compute  d,  we  must  choose  a  value  for  a  ( =  oA  =  oB).  (See  Paragraph  2-2.4  if  no 
other  information  is  available.)  If  we  overestimate  o,  the  consequences  are  two-fold:  first,  we 
overestimate  the  sample  size  n  (=  nA  =  nB)  required,  and  thus  unnecessarily  increase  the  cost  of 
the  test;  but,  by  employing  a  sample  size  that  is  larger  than  necessary,  the  actual  value  of  0(5)  will 
be  somewhat  less  than  we  intended,  which  will  be  all  to  the  good.  On  the  other  hand,  if  we  under- 
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estimate  a,  we  shall  underestimate  the  sample  size  actually  required,  and  by  using  too  small  a 
sample  size,  0 (5)  will  be  somew;hat  larger  than  we  intended,  and  our  chances  of  detecting  real 
differences  when  they  exist  will  be  correspondingly  lessened.  These  effects  of  overestimating  or 
underestimating  a-  ( =  <rA  =  aB)  will  be  similar  in  magnitude  to  those  considered  and  illustrated  in 
Paragraph  3-2.2. 1  for  the  case  of  comparing  the  mean  m  of  a  new  material,  product,  or  process, 
With  a  standard  value  m0. 

As  explained  in  the  preceding  discussion  of  the  Operating  Characteristics  of  the  Test,  an  error 
in  estimating  c  ( =  cA  =  cb)  will  have  no  effect  on  the  significance  level  of  the  test,  provided  that 
the  value  of  is  taken  equal  to  the  100  (1  —  a)  percentile  of  the  f-distribution  for  nA  +  nB  —  2 
degrees  of  freedom,  where  nA  and  nB  are  the  sample  sizes  actually  employed;  and  if  a  A  ^  aB,  the 
effect  will  not  be  serious  provided  that  the  sample  sizes  are  taken  equal. 


3-3.2.2  (Case  2) — Variability  of  A  and  B  it  Unknown,  Cannot  Be  Attumed  Equal. 

Consider  the  data  of  Data  Sample  3-3. 1.2.  Suppose  that  (from  a  consideration  of  the  methods, 
and  not  after  looking  at  the  results)  the  question  to  be  asked  was  whether  the  average  for  Method  A 
exceeded  the  average  for  Method  B. 


Procedure* 

(1)  Choose  a,  the  significance  level  of  the  test. 

(2)  Compute:  XA  and  s\,  XB  and  s|,  from  the 
nA  and  nB  measurements  from  A  and  B. 


(3) 


Compute: 


and 


VB 


Sa 

nB’ 


the  estimated  variances  of  XA  and  XB, 
respectively. 

(4)  Compute  the  "effective  number  of  degrees 
of  freedom” 

/  (Y  a.  +  Pfl)2  o 

"  w  ~  n  ~ 

nA  +  1  ^  nB  +  i 


(5)  Ix>ok  up  for  /'  degrees  of  freedom  in 
Table  A-4,  where /'  is  the  integer  nearest  to 
/;  denote  this  value  by 

(6)  Compute 

U  —  t[ _a  \/rVA  +  Pfl 


*  See  footnote  on  page  3-37. 


Example 

(1)  Let  a  =  .05 

(2)  XA  -  3166.0 

s*x  -  6328.67 

nA  =  4 

XB  -  2240.4 

»  221,661.3 
n*  *=  9 

(3) 

T.  6328.67 

Px  -  — 4  — 

=  1582.17 
Tr  221,661.3 

P*  =  9— 

=  24629.03 


(4) 

f  _  _  (26211.20)* 

J  500652:4  +  60658911.9 
=  11.233  -  2 
=  9.233 

(5)  r  =  9 

t'tt  —  1.833 

(6) 

u  =  1.833  V'26211.20 
=  1.833  (161.90) 

=  296.76 
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Procedure* 

(7)  If  {XA  —  XB)  >  u,  decide  that  the  average 
of  A  exceeds  the  average  of  B;  otherwise, 
decide  that  there  is  no  reason  to  believe 
that  the  average  of  A  exceeds  the  average 
of  B. 

(8)  Let  mA  and  mB  be  the  true  averages  of 
A  and  B.  Note  that  the  interval  from 
J (XA  —  ln)  —  u\  to  oo  is  approximately 
a  one-sided  100  (1  —  a)  %  confidence  in¬ 
terval  estimate  of  the  true  difference 
(mA  -  mB). 


Example 

(7)  5"x  —  XB  =  925.6,  which  is  larger  than  «. 
Conclude  that  the  average  for  Method  A 
exceeds  the  average  for  Method  B. 


(8)  {XA  -  XB )  —  u  =  925.6  -  296.76  = 
628.8.  The  interval  from  628.8  to  »  is 
approximately  a  one-sided  95%  confidence 
interval  estimate  of  the  true  difference  be¬ 
tween  the  averages  for  the  methods. 


3-3.2.3  (Case  3) — Variability  In  Performance  of  Each  of  A  and  B  is  Known  from  Previous  Experience 
and  the  Standard  Deviations  are  aA  and  <rBt  Respectively. 


Data  Sample  3-3. 2.3 

The  observational  data  are  those  of  Data  Sample  3-3.2.1  on  surface  hardness  of  steel  plates.  In 
addition,  it  now  is  assumed  that  the  variability  for  the  two  conditions  was  known  from  previous 
experience  to  be  <rA  =  10.25  and  aB  =  5.00. 


(1)  Choose  a, 

(2)  Look  up  in  Table  A-2. 

(3)  Compute:  XA  and  XB,  the  means  of  the  nA 
and  nB  measurements  from  A  and  B. 


(4)  Compute 


(5)  If  (Xx  —  X B)  >  u,  decide  that  the  average 
of  A  exceeds  the  average  of  B;  otherwise, 
decide  that  there  is  no  reason  to  believe 
that  the  average  of  A  exceeds  the  average 
of  B. 


Example 

Let  a  =  .05 

Zi-a  =  1.645 

XA  -  165 
.t*  =  105 
nA  =  7 
XB  «  155.2 
oi  =  25 
71/,  =  5 

u  =  1.645  VI 5  +  5 
=  L645  (4.472) 

=  7.4 

{XA  —  XB)  —  9.8,  which  is  larger  than  u. 
Conclude  that  the  average  hardness  for 
Condition  A  exceeds  the  average  hardness 
for  Condition  B. 


[Ono-«ided  Normal  Test] 

Procedure 

the  significance  level  of  the  test.  (1) 

(2) 

(3) 


(4) 


(5) 


(6)  Let  rax  and  m«  be  the  true  averages  of 
A  and  B.  Note  that  the  interval  from 
|(Xx  -  XB)  — uj  to  <«  is  a  1  —  a  one¬ 
sided  confidence  interval  estimate  of  the 
true  difference  (mA  —  mB). 


(6)  The  interval  from  2.4  to  »  is  a  95%  one¬ 
sided  confidence  interval  estimate  of  the 
true  difference  between  averages. 


*  See  footnotes,  and  also  the  discussion  of  the  properties  and  limitations  of  this  type  of  procedure,  in  Paragraph  3-3.1.2. 
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Operating  Characteristics  of  the  Test.  Figures  3-7  and  3-8  give  the  operating  characteristic  «/C) 
curves  of  the  above  test  for  a  =  .05  and  a  =  .01,  respectively,  for  various  values  of  n. 

If  nA  =  nB  -  n  and  {mA  —  mB)  is  the  true  positive  difference  between  the  averages,  then  putting 

j  _  (mA  -  mB) 

~ 

we  can  read  j8,  the  probability  of  failing  to  detect  a  difference  of  size  (mA  —  mB). 

If  nA  =  cnB,  we  can  put 

d  _  (mA  -  mB) 

Voa  +  C<T*b 

and  again  read  £),  the  probability  of  failing  to  detect  a  difference  of  size  (mA  —  mB). 

Selection  of  Sample  Size.  We  choose 
a,  the  significance  level  of  the  test 

/S,  the  probability  of  failing  to  detect  a  difference  of  size  (mA  —  mB). 

If  we  wish  nA  =  nB  =  n,  we  compute 

,  _  (wx  -  mB) 

V<ri  +  Ob 

and  we  may  use  Table  A-9  directly  to  obtain  the  required  sample  size  n. 

If  we  wish  to  have  nA  and  nB  such  that  nA  =  cnB,  then  we  may  compute 

,  =  (mA  -  mB) 

a\  +  Cob 

and  use  Table  A-9  to  obtain  n  =  nA. 


3-3. 2.4  (Cate  4) — The  Observations  are  Paired. 

Often,  an  experiment  is,  or  can  be,  designed  so  that  the  observations  are  taken  in  pairs.  The  two 
units  of  a  pair  are  chosen  in  advance  so  as  to  be  as  nearly  alike  as  possible  in  all  respects  other  than 
the  characteristic  to  be  measured,  and  then  one  member  of  each  pair  is  assigned  at  random  to 
Treatment  A,  and  the  other  to  Treatment  B.  For  a  discussion  of  the  advantage  of  this  approach, 
see  Paragraph  3-3. 1.4. 


Data  Sample  3-3.2 .4 — Molecular  Weight  of  Dextrons 

During  World  War  II  bacterial  polysaccharides  (dextrons)  were  considered  and  investigated  for 
use  as  blood  plasma  extenders.  Sixteen  samples  of  hydrolyzed  dextrons  were  supplied  by  various 
manufacturers  in  order  to  assess  two  chemical  methods  for  determining  the  average  molecular 
weight  of  dextrons. 


Method  A 

Method  B 

=  X*  -X, 

62,700 

56,400 

6,300 

29,100 

27,500 

1,600 

44,400 

42,200 

2,200 

47,800 

46,800 

1,000 

36,300 

33,300 

3,000 

40,000 

37,100 

2,900 

43,400 

37,300 

6,100 

35,800 

36,200 

-  400 
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Method  A 

Method  B 

II 

? 

i 

* 

33,900 

35,200 

-1,300 

44,200 

38,000 

6,200 

34,300 

32,200 

2,100 

31,300 

27,300 

4,000 

38,400 

36,100 

2,300 

47,100 

43,100 

4,000 

42,100 

38,400 

3,700 

42,200 

39,900 

2,300 

Procedure 

Example 

(1) 

Choose  a,  the  significance  level  of  the  test.  (1) 

Let  a  =  .05 

(2) 

Compute  the  Xd  and  sd  for  the  n  differ-  (2) 

Xd  =  2875 

ences,  Xd.  Each  Xd  represents  an  observa¬ 

sd  =  2182.2 

tion  on  A  minus  the  paired  observation  on 

B. 

n  =  16 

(3) 

Look  up  for  n  —  1  degrees  of  freedom  (3) 

t.n  for  15  d.f.  -  1.753 

in  Table  A-4. 

(4) 

Compute 

(4) 

sd 

.  _co  /2182.2N 

U~ll-Vn 

«  -  1-753  ^  4  j 

-  1.753  (545.6) 

=  956.4 

(5) 

If  Xd  >  u,  decide  that  the  average  of  A  (5) 

Xd  =  2875,  which  is  larger  than  u.  Con¬ 

exceeds  that  of  B;  otherwise,  there  is  no 

clude  that  the  average  for  Method  A 

reason  to  believe  the  average  of  A  exceeds 

exceeds  the  average  for  Method  B. 

that  of  B. 

(6) 

Note  that  the  open  interval  from  Xd 

-  w  (6) 

Xd  -  u  «  (2875  -  956)  =  1919.  The  in¬ 

to  -f  oo  is  a  one-sided  100  (!—«)% 

con- 

terval  from  1919  to  +  «  is  a  one-sided 

fidence  interval  for  the  true  difference 

95%  confidence  interval  for  the  true  dif¬ 

(mA  -  mB). 

ference  between  the  averages  of  the  two 

methods. 


Operating  Characteristics  of  the  Test.  Figures  3-5  and  3-6  give  the  operating  characteristic  (OC) 
curves  of  the  test  for  «  =  .05  and  a  =  .01,  respectively,  for  various  values  of  n,  the  number  of  pairs 
involved. 

Choose: 

8  =  (mA  —  mB),  the  true  difference  between  the  averages  (unknown,  of  course) 

Some  value  of  <r  ( =  t rd),  the  true  standard  deviation  of  a  signed  difference  Xd. 

(We  may  use  an  estimate  from  previous  data.  If  OC  curve  is 
consulted  after  the  experiment,  we  may  use  the  estimate  from 
the  experiment,) 

Compute 


cr 
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We  can  then  read  from  the  OC  curve  for  a  given  significance  level  a.  and  number  of  pairs  n,  a  value 
of  /3(a).  The  /3(5)  read  from  the  curve  is  /S(a  |  <r,  a,  n ),  i.e.,  0(5,  given  <j ,  a,  n) — the  probability  of  failing 
to  detect  a  difference  (mA  -  rnB)  of  magnitude  +a  when  the  given  test  is  carried  out  with  n  pairs, 
at  the  a-level  of  significance,  and  the  population  standard  deviation  of  the  differences  Xd  actually 
t8  o. 

If  we  use  too  large  a  value  for  a,  the  effect  is  to  underestimate  d,  and  consequently  to  overestimate 
0(a),  the  probability  of  not  detecting  a  difference  (mA  —  mB)  of  size  +5  when  it  exists.  Conversely, 
if  we  choose  too  small  a  value  of  <r,  then  we  shall  overestimate  d  and  underestimate  0(5).  The  true 
value  of  0(a)  is  determined,  of  course,  by  the  actual  number  of  pairs  n,  the  significance  level  a 
employed,  and  the  true  value  of  a  ( =  <rd). 

Selection  of  Number  of  Pairs  ( n ).  If  we  choose 

a  =  (mA  -  mB ),  the  value  of  the  (positive)  average  difference  that  we  desire  to  detect 
a,  the  significance  level  of  the  test 
0,  the  probability  of  failing  to  detect  a  difference  of  +5 


and  compute 

d  =  (mA  -  m„) 
a 


where  a  (=  <rd)  is  the  standard  deviation  of  the  population  of  signed  differences  Xd  of  the  type 
concerned,  then  we  may  use  Table  A-9  to  obtain  a  good  approximation  to  the  required  number  of 
pairs  n.  If  we  take  a  =  .01,  then  we  must  add  3  to  the  table  value.  If  we  take  a  =  .05,  then  we 
must  add  2  to  the  table  value.  (In  order  to  compute  d,  we  must  choose  a  value  for  a.) 

If,  when  planning  the  test,  we  overestimate  a ,  the  consequences  are  two-fold:  first,  we  over¬ 
estimate  the  number  of  pairs  required,  and  thus  unnecessarily  increase  the  cost  of  the  test;  but,  by 
employing  a  sample  size  that  is  larger  than  necessary,  the  actual  value  of  0(5)  will  be  somewhat  less 
than  we  intended,  which  will  be  all  to  the  good.  On  the  other  hand,  if  we  underestimate  a,  we  shall 
underestimate  the  number  of  pairs  actually  required,  and  by  using  too  small  a  sample  size,  0(5)  will 
be  somewhat  larger  than  we  intended,  and  our  chances  of  detecting  real  differences  when  they  exist 
will  be  correspondingly  lessened. 


Finally,  it  should  be  noted,  that  inasmuch  as  the  test  criterion  u  =  1 ^7=  does  not  depend  on  <7, 

an  error  in  estimating  <r  when  planning  the  test  will  not  alter  the  level  of  significance,  which  will  be 
precisely  equal  to  the  value  of  a  desired,  provided  that  is  taken  equal  to  the  100  (1  —  a)  per¬ 
centile  of  the  ^distribution  for  n  —  1  degrees  of  freedom,  where  n  is  the  number  of  pairs  actually 
employed. 


3-4  COMPARING  THE  AVERAGES  OF  SEVERAL  PRODUCTS 

Do  the  averages  of  t  products  l,  2, ....  t  differ?  We  shall  assume  that  n\  =  n,  » _ =  nt  =  n. 

If  the  n’s  are  in  fact  not  all  equal,  but  differ  only  slightly,  then  in  the  following  procedure  we  may 
replace  n  by  the  harmonic  mean  of  the  n’s, 


n»  =  f/d/tti  4-  1/nj  4-  .  .  .  +  1/n,) 
and  obtain  a  satisfactory  approximation. 


3-40 


COMPARING  AVERAGE  PERFORMANCE 


AMCP  706-110 


Data  Sample  3-4 — Breaking-strength  of  Cement  Briquettes 

The  following  data  relate  to  breaking-strength  of  cement  briquettes  (in  pounds  per  square  inch). 
The  question  to  be  answered  is:  Does  the  average  breaking-strength  differ  for  the  different  groups? 


Group 

1 

2 

3 

4 

5 

518 

508 

554 

555 

536 

560 

574 

598 

567 

492 

538 

528 

579 

550 

528 

510 

534 

538 

535 

572 

544 

538 

544 

540 

506 

zXi 

2670 

2682 

2813 

2747 

2634 

Tli 

5 

5 

5 

5 

5 

Xi 

534.0 

536.4 

562.6 

549.4 

526.8 

1427404 

1440924 

1585141 

1509839 

1391364 

(2X)* 

n 

1425780 

1438624.8 

1582593.8 

1509201.8 

1387591.2 

n 

1624 

2299.2 

2547.2 

637.2 

3772.8 

s2 

406 

574.8 

636.8 

159.3 

1 

943.2 

Excerpted  with  permission  from  Stati&tieal  Exercute,  Part  If,  Analysts  of  Varianca  and  AitociaUd  Tochniquto,  by  N,  L.  Johnson,  Copyright,  1967 
Department  of  Statistic!,  University  College,  London. 


Procedure 

(1)  Choose  a,  the  significance  level  (the  risk  of  (1)  Let 
concluding  that  the  averages  differ,  when 

in  fact  all  averages  are  the  same). 

(2)  Compute:  (2) 

sf ,  ^  .  ,  s2  . 


(3)  Compute  (3) 

si  =  ^(si +  $?  +  •••  + s^) 

If  the  «i  are  not  all  equal,  the  following 
formula  usually  is  to  be  preferred: 

s1,  = 

(nt  -  1)  s?  +  (n2  -  1)  4  +  .  . .  +  (n,  —  1)  s] 

(nt  +  n2  -f- . . .  +  n,)  —  t 


Example 

a  «=  .01 


si  -  406.0 
4  -  574.8 
sj  =  636.8 
s?  =  159.3 
sj  =  943.2 

.  2720.1 

*'  "_5“ 
=  544.0 


s.  =  23.32 
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Procedure 

Example 

Look  up  (f,  r)  in  Table  A-10  where 

(4) 

v  ■=  («i  +  n*  +  . . 

•  •  +  W()  —  t. 

p  -  25  —  5 

-  20 

t  =  5 

q.tt  (5,20)  =  5.29 

Compute 

(5) 

<Zl-a  s. 

5.29  (23.32) 

W~  Vn 

V5 

123.36 

"  2.236 

=  55.2 

(6)  If  the  absolute  difference  between  any*two 
sample  means  exceeds  w,  decide  that  the 
averages  differ;  otherwise,  decide  that  there 
is  no  reason  to  believe  the  averages  differ. 


(6)  The  greatest  difference  between  sample 
means  is  562.6  —  526.8  =  35.8,  which  is 
less  than  w.  We,  therefore,  have  no  reason 
to  believe  that  the  group  averages  differ. 


Note:  It  is  worth  noting  that  we  simultaneously  can  make  confidence  interval  estimates  for  each  of 

the  ^  pairs  of  differences  between  product  averages,  with  a  confidence  of  1  —  a  that  all  of  the 

estimates  are  correct.  The  confidence  intervals  are  (X <  —  X3)  dt  w,  where  X„  Xh  are  sample  means 
of  the  ith  and  ;lh  products. 
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CHAPTER  4 

COMPARING  MATERIALS  OR  PRODUCTS  WITH  RESPECT  TO 
VARIABILITY  OF  PERFORMANCE 

4-1  COMPARING  A  NEW  MATERIAL  OR  PRODUCT  WITH  A  STANDARD 
WITH  RESPECT  TO, VARIABILITY  OF  PERFORMANCE 


The  variability  of  a  standard  material,  product,  or  process,  as  measured  by  its  standard  deviation, 
is  known  to  be  <r0-  We  consider  the  following  three  problems: 

(a)  Does  the  variability  of  the  new  product  differ  from  that  of  the  standard?  See  Paragraph 
4-1.1. 

(b)  Does  the  variability  of  the  new  product  exceed  that  of  the  standard?  See  Paragraph  4-1.2. 

(c)  Is  the  variability  of  the  new  product  less  than  that  of  the  standard?  See  Paragraph  4-1.3. 

It  is  important  to  decide  which  of  the  three  problems  is  appropriate  before  taking  the  observa¬ 
tions.  If  this  is  not  done,  and  the  choice  of  problem  is  influenced  by  the  observations,  both  the 
significance  level  of  the  test  (i.e.,  the  probability  of  an  Error  of  the  First  Kind)  and  the  operating 
characteristics  of  the  test  may  differ  considerably  from  their  nominal  values. 

The  tests  given  are  exact  when: 

(a)  the  observations  for  an  item,  product,  or  process  are  taken  randomly  from  a  single  population 
of  possible  observations;  and, 

(b)  within  the  population,  the  quality  characteristic  measured  is  normally  distributed. 

4-1.1  DOES  THE  VARIABILITY  OF  THE  NEW  PRODUCT  DIFFER  FROM.  THAT  OF  THE  STANDARD? 

The  variability  in  the  performance  of  a  standard  material,  product,  or  process,  as  measured  by 
its  standard  deviation,  is  known  to  be  cru.  We  wish  to  determine  whether  a  given  item  differs  in 
variability  of  performance  from  the  standard.  We  wish,  from  analysis  of  the  data,  to  make  one 
of  the  following  decisions: 

(a)  The  variability  in  performance  of  the  new  product  differs  from  that  of  the  standard. 

(b)  There  is  no  reason  to  believe  the  variability  of  the  new  product  is  different  from  that  of  the 
standard. 


Data  Sample  4-1.1 — Capacity  of  Batteries 

The  standard  deviation  <70  of  capacity  for  batteries  of  a  standard  type  is  known  to  be  1.66  ampere 
hours.  The  following  capacities  (ampere  hours)  were  recorded  for  10  batteries  of  a  new  type: 
146,  141,  135,  142,  140,  143,  138,  137,  142,  136. 

We  wish  to  compare  the  new  type  of  battery  with  the  standard  type  with  regard  to  variability 
of  capacity.  The  question  to  be  answered  is:  Does  the  new  type  differ  from  the  standard  type 
with  respect  to  variability  of  capacity  (either  a  decrease  or  an  increase  is  of  interest)? 
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Procedure 


Example 


(1) 

Choose  a,  the  level  of  significance  of  the 
test. 

(1) 

Let  a  =  .05 

tt) 

Look  up  Bu  and  BL  both  for  n  —  1  degrees 

(2) 

n  -  1  =  9 

of  freedom  in  Table  A-20. 

Bu  for  9  d.f.  =  1.746 
Bi  for  9  d.f.  =  .6657 

(3) 

Compute  s,  from  the  n  observations 

(3) 

s 


4 


ZX*  -  (£X)*/n 
n  —  1 


(4)  Compute: 

sL  =  B,.  s 


Sy  =  By  S 


(4) 


s 


4 


196108  -  196000 
9 


= 

4f 

Vl2 

3.464 

Sl  = 

(.6657) 

(3.464) 

= 

2.31 

Bu  = 

(1.746) 

(3.464) 

= 

6.05 

(5)  If  c0  does  not  lie  between  sL  and  sUt  decide 
that  the  variability  in  performance  of  the 
new  product  differs  from  that  of  the  stand¬ 
ard;  otherwise,  that  there  is  no  reason  to 
believe  the  new  product  differs  from  the 
standard  with  regard  to  variability. 

(6)  It  is  worth  noting  that  the  interval  from  sL 
to  sy  is  a  100  (1  —  a)  %  confidence  interval 
estimate  of  o,  the  standard  deviation  of  the 
new  product.  (See  Par.  2-2.3. 1). 


(5)  Since  co  «=  1.66  does  not  lie  between  the 
limits  2.31  to  6.05,  conclude  that  the  vari¬ 
ability  for  the  new  type  does  differ  from  the 
variability  for  the  standard  type. 


(6)  The  interval  from  2.31  to  6.05  ampere 
hours  is  a  95%  confidence  interval  estimate 
for  the  standard  deviation  of  the  new  type. 


Operating  Characteristics  of  the  Test.  Operating-characteristic  (OC)  curves  for  this  Neyman- 
Pearson  “unbiased  Type  A”  test  of  the  null  hypothesis  that  c  =  c0  relative  to  the  alternative  that 
c  ^  co  are  not  currently  available  except  for  two  special  cases  considered  in  the  original  Neyman- 
Pearson  memoir.'"  These  special  cases  and  more  general  considerations  indicate  that  the  OC 
curves  for  this  test  will  not  differ  greatly,  except  for  the  smallest  sample  sizes,  from  the  OC  curves 
for  the  corresponding  traditional  "equal-tail”  test  (see  Figures  6.15  and  6.16  of  Bowker  and  Lieber- 
man12').  The  OC  curve  for  the  present  test  for  a  given  significance  level  and  sample  size  n  will  lie 
above  the  OC  curve  of  the  corresponding  “equal-tail”  test  for  a  >  c0  and  below  the  OC  curve  for  the 
"equal-tail”  test  for  a  <  <ro.  In  other  words,  the  chances  of  failing  to  detect  that  a  exceeds  co  are 
somewhat  greater  with  the  present  test  than  with  the  “equal-tail”  test,  and  somewhat  less  of  failing 
to  detect  that  c  is  less  than  cr«.  The  reader  is  reminded,  however,  that  if  there  is  special  interest  in 
determining  whether  a  >  c0,  or  special  interest  in  determining  whether  a  <  co,  the  problem  and 
procedure  of  this  Paragraph  is  not  at  all  appropriate,  and  Paragraph  4-1.2  or  4-1.3  should  be 
consulted. 
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4-1.2  DOES  THE  VARIABILITY  OF  THE  NEW  PRODUCT  EXCEED  THAT  OF  THE  STANDARD? 

The  variability  in  performance  of  a  standard  material,  product,  or  process,  as  measured  by  its 
standard  deviation,  is  known  to  be  <r0.  We  wish  to  determine  whether  the  variability  in  perform¬ 
ance  of  a  new  product  exceeds  that  of  the  standard.  We  wish,  from  analysis  of  the  data,  to  make 
one  of  the  following  decisions: 

(a)  The  variability  in  performance  of  the  new  product  exceeds  that  of  the  standard. 

(b)  There  is  no  reason  to  believe  the  variability  of  the  new  product  exceeds  that  of  the  standard. 

In  terms  of  Data  Sample  4-1.1,  let  us  suppose  that — in  advance  of  looking  at  the  data! — the 
important  question  is:  Does  the  variability  of  the  new  type  exceed  that  of  the  standard? 


Procedure 

(1)  Choose  «,  the  level  of  significance  of  the 
test. 

(2)  Look  up  A«  for  n  —  1  degrees  of  freedom  in 
Table  A-21. 

(3)  Compute  s,  from  the  n  observations. 

(4)  Compute  sL  =  Aa  s 

(5)  If  sL  exceeds  cB,  decide  that  the  variability 
of  the  new  product  exceeds  that  of  the 
standard;  otherwise,  that  there  is  no  reason 
to  believe  that  the  new  product  exceeds  the 
standard  with  regard  to  variability. 

(6)  It  is  worth  noting  that  the  interval  above 
sf,  is  a  100  (1  —  «)  %  confidence  interval 
estimate  of  <r,  the  standard  deviation  of  the 
new  product.  (See  Par.  2-2.3.2). 


Example 

(1)  Let  «  =  .05 

(2)  n  -  1  =  9 

A. 06  for  9  d.f.  =  .7293 

(3)  s  =  3.464 

(4)  sL  =  .7293  (3.464) 

=  2.53 

(5)  Since  2.53  exceeds  1.66,  conclude  that  the 
variability  of  the  new  type  exceeds  that  of 
the  standard  type. 


(6)  The  interval  from  2.53  to  +  °o  is  a  95% 
confidence  interval  estimate  of  the  stand¬ 
ard  deviation  of  the  new  type. 


Operating  Characteristics  of  the  Test.  Figure  4-1  provides  operating-characteristic  (OC)  curves 
of  the  test  for  a  =  0.05  and  various  values  of  n.  Let  <7!  denote  the  true  standard  deviation  of  the 
new  product.  Then  the  OC  curves  of  Figure  4-1  show  the  probability  0  =  0  (X|.05,  n)  of  failing 
to  conclude  that  c,  exceeds  <70  when  <n  =  Xo-0  and  the  test  is  carried  out  at  the  a  =  0.05  level  of 
significance  using  a  value  of  s  derived  from  a  sample  of  size  n.  Similar  OC  curves  for  the  rase  of 
a  =  0.01  are  given  in  Figure  6.18  of  Bowker  and  Lieberman.<2)  OC  curves  are  easily  constructed 
for  other  values  of  n  —  and,  if  desired,  other  values  of  a  ~  by  utilizing  the  fact  that  if  the  test  is 
conducted  at  the  a  level  of  significance  using  a  value  of  s  based  on  a  sample  of  size  n,  then  the 
probability  of  failing  to  conclude  that  <n  exceeds  <ra  when  <7,  -  X<r0  is  exactly  &  for 

X  =  X  («,  /3,  n)  =  VxU  (n  -  D/xHn  -  1), 

where  (v)  is  the  P-probability  level  of  x2  for  v  degrees  of  freedom,  as  given  in  Table  A-3.  Values 
of  P  (a,  p,  Hi)  =  X5  (a,  p,  n )  corresponding  to  a  =  0.05  and  a  =  0.01,  for  p  =  0.005,  0.01,  0.025, 
0.05,  0.10,  0.25,  0.50,  0.75,  0.90,  0.95,  0.975,  0.99,  and  0.995  are  given  in  Tables  8.1  and  8.2  of 
Eisenhart'3’  for  n,  =  n  -  1  =  1(1)30(10)100,  120,  ®. 
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Figure  4-2 .  Operating  characteristics  of  the  one-sided  x2-test  to  determine  whether  the  standard  deviation 
ox  of  a  new  product  exceeds  the  standard  deviation  o0  of  a  standard  ( a  =  .05). 

Adapted  with  permission  from  AnnaU  of  MaOumaiical  Statulieo,  Vol.  17,  No.  2,  June  1946,  from  article  entitled  “Operating  Characteristics  for  the 
Common  Statistical  Tests  of  Significance’  by  C.  D.  Ferris,  F.  E.  Grubbs,  and  C.  L.  Weaver. 


Selection  of  Sample  Size.  If  we  choose 


a,  the  significance  level  of  the  test 

and,  0,  the  probability  of  failing  to  detect  that  ci  exceeds  <r 9  when  o-,  =  X<r0 

then  for  a  =  0.05  we  may  use  the  OC  curves  of  Figure  4-1  to  determine  the  necessary  sample 
size  n. 

Example:  Choose 
X  =  f-‘  =  1.5 

<70 

a  —  0.05 
0  «  0.05 

then  from  Figure  4-1  it  is  seen  that  n  =  30  is  not  quite  sufficient,  and  n  -  40  is  more  than  sufficient. 
Visual  interpolation  suggests  n  —  35. 

Alternatively,  one  may  compute  the  necessary  sample  size  from  the  approximate  formula 

„  /  o  >,\  1  i  1  (Z\-a  +  X  •  Zl_jV 

n  =  n  («,  0,  X)  =1+2  l — - ) 
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where  zp  is  the  P-probability  point  of  the  standard  normal  variable  z,  values  of  which  are  given  in 
Table  A-2  for  various  values  of  P.  Thus,  in  the  foregoing  example  we  find 

1,1  /1.645  +  (1.5)  (1.645) Y 
n  “  1  +  2\  1.5-1  ) 

-  1  +  \  (^5-)*  =  1  +  \  (8.225) 1  =  1  +|  (67.65) 

=  34.8 

which  rounds  to  n  =  35.  Chand(4)  has  found  this  formula  generally  quite  satisfactory,  and  that 
“even  for  such  a  small  value  as  n  =  5“  it  “errs  on  the  safe  side  in  the  sense  that  it  gives  (at  least 
for  a  =  0)  a  sample  size  which  will  always  be  sufficient.” 

Check;  For  n  =  35, 


A  (.05,  .05,  35,  -  -  Jgf  -  \/2.2G 


=  1.50 


Hence  0  =  0.05  for  X  =  1.50. 

4-1.3  IS  THE  VARIABILITY  OF  THE  NEW  PRODUCT  LESS  THAN  THAT  OF  THE  STANDARD? 

The  variability  in  performance  of  a  standard  material,  product,  or  process,  as  measured  by  its 
standard  deviation,  is  known  to  be  <r0.  We  wish  to  determine  whether  the  variability  in  perform¬ 
ance  of  the  new  product  is  less  than  that  of  the  standard.  We  wish,  from  analysis  of  the  data,  to 
make  one  of  the  following  decisions: 

(a)  The  variability  in  performance  of  the  new  product  is  less  than  that  of  the  standard. 

(b)  There  is  no  reason  to  believe  the  variability  in  performance  of  the  new  product  is  less  than 
that  of  the  standard. 


Data  Sampla  4-1.3 — Cutoff  Bias  of  Tubes 

A  manufacturer  has  recorded  the  cutoff  bias  of  a  sample  of  ten  tubes,  as  follows  (volts): 

12.1,  12.3,  11.8,  12.0,  12.4,  12.0,  12.1,  11.9,  12.2,  12.2. 

The  variability  of  cutoff  bias  for  tubes  of  a  standard  type  as  measured  by  the  standard  deviation 
is  cr0  =  0.208  volt. 

Let  us  assume  with  respect  to  Data  Sample  4-1.3  that  the  important  question  is:  Is  the  variability 
of  the  new  type  with  respect  to  cutoff  bias  less  than  that  of  the  standard  type? 


Procoduro 

ExampU 

(1) 

Choose  a,  the  level  of  significance  of  the 

(1) 

Let  a  =  .05 

test. 

(2) 

Look  up  Ai_<,  for  n  —  1  degrees  of  freedom 

(2) 

n  —  1  =  9 

in  Table  A-21. 

A,n  for  9  d.f.  =  1.645 

(3) 

Compute  s,  from  the  n  observations 

(3) 

„  _  fax*  -  (zxy/n 

s  =  JJf 

\  n  -  1 

\  9 

=  V .  0333 

=  .1826 
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Procedure 

(4)  Compute  Sv  =  Aj_a  s 

(5)  If  Su  is  less  than  <r0,  decide  that  the  vari¬ 
ability  in  performance  of  the  new  product 
is  less  than  that  of  the  standard ;  otherwise, 
that  there  is  no  reason  to  believe  the  new 
product  is  less  variable  than  the  standard. 

(6)  It  is  worth  noting  that  the  interval  below  sL, 
is  a  100  (1  —  a)  %  confidence  interval  esti¬ 
mate  of  a,  the  standard  deviation  of  the 
new  product.  (See  Par.  2-2. 3. 2.) 


Exomple 

(4)  sv  =  1.645  (.1826) 

=  0.300 

(5)  Since  .300  is  not  less  than  .208,  conclude 
that  there  is  no  reason  to  believe  that  the 
new  type  is  less  variable  than  the  standard. 


(6)  The  interval  below  0.300  is  a  95%  confi¬ 
dence  interval  estimate  of  the  standard 
deviation  of  the  new  type. 


Operating  Characteristics  of  the  Test.  Figure  4-2  provides  operating-characteristic  (OC)  curves 
of  the  test  for  a  =  0.05  and  various  values  of  n.  Let  <ri  denote  the  true  standard  deviation  of  the 
new  product.  Then  the  OC  curves  of  Figure  4-2  show  the  probability  0  =  0  (X  |  .05,  n)  of  failing 
to  conclude  that  o\  is  less  than  <r0  when  ax  =  A<r«  and  the  test  is  carried  out  at  the  a  =  0.05  level 
of  significance  using  a  value  of  s  derived  from  a  sample  of  size  n.  Similar  OC  curves  for  the  case 
of  a  -  0.01  are  given  in  Figure  6.20  of  Bowker  and  Lieberman. t!)  OC  curves  are  easily  constructed 


Figure  i-2.  Operating  characteristics  of  the  one-sided  x--test  to  determine  whether  the  standard  deviation 
<7i  of  a  new  product  is  less  than  the  standard  deviation  <r0  of  a  standard  (a  =  .05). 

Adapted  with  permission  from  Annal «  0/  M alhznatical  StalUlirt,  Vol.  17,  No.  2.  June  194S,  from  article  entitled  "Operating  Characteriatica  for  the 
Common  Statistical  Testa  of  Pignifieanre"  by  O.  D.  Ferris,  F.  E,  Grubbs,  and  C.  L.  Weaver. 
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for  other  values  of  n  —  and,  if  desired,  other  values  of  a  —  by  utilizing  the  fact  that  if  the  test  is 
conducted  at  the  a  level  of  significance  using  a  value  of  s  based  on  a  sample  of  size  n,  then  the 
probability  of  failing  to  conclude  that  <n  is  less  than  <rn  when  ax  =  X<n>  is  exactly  0  for 

X  =  X  (a,  0,  n)  =  V x-  (n  -  l)/x?_,j  (n  -  1) , 

where  Xp  {v)  is  the  P-probability  level  of  x2  for  v  degrees  of  freedom,  as  given  in  Table  A-3. 


Selection  of  Sample  Size.  If  we  choose 


a,  the  significance  level  of  the  test 

and,  B,  the  probability  of  failing  to  detect  that  <rx  is  less  than  <r0  when  =  X<r0 

then  for  a  =  0.05  we  may  use  the  OC  curves  of  Figure  4-2  to  determine  the  necessary  sample  size  n. 

Example:  Choose 

*  -  2  "  °'5 
a  =  0.05 
B  =  0.05 

then  from  Figure  4-2  it  is  seen  that  n  =  10  is  not  quite  sufficient,  and  n  =  15  is  more  than  sufficient. 
Visual  interpolation  suggests  n  =  14. 

Alternatively,  one  may  compute  the  necessary  sample  size  from  the  approximate  formula 

„  /  a  i  i  I  (* !-«  +  X  ■  2i_dV 

n  =  n  (a,  P,  X)  =  1  +  2  1  ---  — ^ - I 

where  zr  is  the  P-probability  point  of  the  standard  normal  variable  z,  values  of  which  are  given  in 
Table  A-2  for  various  values  of  P.  Thus,  in  the  foregoing  example  we  find 

,  ,  1  /l. 645  +  (0.5)  (1.645) V 
n  " ‘  1  +  2  V  1  -  0(5  / 

=  1  +  g  (204657°)2  =  1  +  \  (4.935) 2  =  l+\  (24.35) 

=  13.18 

which  rounds  to  n  =  13. 

Check:  For  n  =  13, 

X  (.05,  .05.  IS,  -  -  V0.2487 

-  0.499  <  0.50 
Hence,  fi  *=  0.05  for  X  =  0.50 . 
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4-2  COMPARING  TWO  MATERIALS  OR  PRODUCTS  WITH  RESPECT  TO 
VARIABILITY  OF  PERFORMANCE 


We  consider  two  problems: 

(a)  Does  the  variability  of  product  A  differ  from  that  of  product  B?  (We  are  not  concerned 
which  is  larger).  See  Paragraph  4-2.1. 

(b)  Does  the  variability  of  product  A  exceed  that  of  product  B?  See  Paragraph  4-2.2. 

It  is  important  to  decide  which  of  these  two  problems  is  appropriate  before  taking  the  observa¬ 
tions.  If  this  is  not  done,  and  the  choice  of  problem  is  influenced  by  the  observations,  both  the 
significance  level  of  the  test  (i.e.,  the  probability  of  an  Error  of  the  First  Kind)  and  the  operating 
characteristics  of  the  test  may  differ  considerably  from  their  nominal  values.  The  tests  given  are 
exact  when: 

(a)  the  observations  for  an  item,  product,  or  process  are  taken  randomly  from  a  single  population 
of  possible  observations;  and, 

(b)  within  the  population,  the  quality  characteristic  measured  is  normally  distributed. 

In  the  following,  it  is  assumed  the  appropriate  problem  is  selected  and  then  nA,  nH  observations 
are  taken  from  items,  processes,  or  products  A  and  B,  respectively. 


4-2.1  DOES  THE  VARIABILITY  OF  PRODUCT  A  DIFFER  FROM  THAT  OF  PRODUCT  B? 

We  wish  to  test  whether  the  variability  of  performance  of  two  materials,  products,  or  processes 
differ,  and  we  are  not  particularly  concerned  which  is  larger.  We  wish,  from  analysis  of  the  data, 
to  make  one  of  the  following  decisions: 

(a)  The  two  products  differ  with  regard  to  their  variability. 

(b)  There  is  no  reason  to  believe  the  two  products  differ  with  regard  to  their  variability. 


Data  Sample  4-2.1 — Dive-bombing  Methods 

The  performance  of  each  of  two  different  dive-bombing  methods  is  measured  a  dozen  times  with 
the  following  results: 


Method  A 

Method  B 

52G 

414 

406 

430 

499 

419 

627 

453 

585 

504 

459 

459 

415 

337 

460 

598 

506 

425 

450 

438 

624 

456 

506 

385 

Let  us  suppose  that,  in  the  case  of  Data  Sample  4-2.1,  the  question  to  be  answered  is:  Do  the  two 
methods  differ  in  variability  (it  being  of  interest  if  either  is  more  variable  than  the  other)? 
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Procedure 

(1)  Choose  a,  the  level  of  significance  of  the 
test. 

(2)  Look  up  F i_«/2  for  (n.,  —  1,  nB  —  1)  de¬ 
grees  of  freedom,  and  for  ( nB  -  1, 

n*  —  1)  degrees  of  freedom,  in  Table  A-5. 

(3)  Compute  and  Sb  from  the  observations 
from  A  and  B,  respectively. 

(4)  Compute  F  =  s\/s*B 

(5)  If  F  >  F,_a/,  ( nA  -  1,  nB  -  1)  or 

p  < - A - - - 

Fi-c.ii  (nB  -  1,  nA  -  1) 

decide  that  the  two  products  differ  with 
regard  to  their  variability;  otherwise,  there 
is  no  reason  to  believe  that  they  differ. 

(6)  It  is  worth  noting  that  the  interval  between 

_! _ 

F |_/*  {nA  -  1,  n„  -  1)  Vsy 

and 

Fi-.,i  (nB  -  1  ,nA  -  1) 

is  a  100  (1  —  a)  %  confidence  interval  esti¬ 
mate  of  the  ratio 


Example 

(1)  Let  a  =  .05 

(2)  nA  —  1  =  11 

Its  —  1  =  11 

F.tn  (11,  11)  =  3.48 

(3)  A  =  5545 

Sg  =  4073 

(4)  F  =  5545/4073 

=  1.36 

(5)  F.m  (11,  11)  =  3.48 

=  0,29 

Since  F  is  not  larger  than  3.48,  and  is  not 
smaller  than  0.29,  there  is  no  reason  to 
believe  that  the  two  bombing  methods 
differ  in  variability. 

(6)  The  interval  between  0.39  (i.e.,  0.29  X 
1.36)  and  4.73  (i.e.,  3.48  X  1.36)  is  a  95% 
confidence  interval  estimate  of  the  ratio  of 
the  true  variances,  cta/<jb- 


Operating  Characteristics  of  the  Test.  Operating-characteristic  (OC)  curves  for  this  traditional 
“equal-tail”  test  of  the  null  hypothesis  that  <rA  =  <rB  relative  to  the  alternative  <rA  ^  <rB  are  given 
in  Figures  7.1  and  7.2  of  Bowker  and  Lieberman<2)  for  the  case  of  equal  sample  sizes  nA  -  nB  —  n, 
and  significance  levels  a  =■  0.05  and  a  =  0.01,  respectively.  These  curves  may  be  used  to  deter¬ 
mine  the  common  sample  size  nA  =  nB  -  n  needed  to  achieve  a  preassigned  risk  0  of  failing  to 
detect  that  <rA/<rB  =  X  when  the  test  is  carried  out  at  the  a  —  0.05  or  a  -  0.01  level  of  significance. 
The  reader  is  reminded,  however,  that  if  there  is  special  interest  in  determining  whether  <rA  >  <rB, 
the  problem  and  procedure  of  this  Paragraph  is  not  at  all  appropriate,  and  Paragraph  4-2.2  should 
be  consulted. 


4-2.2  DOES  THE  VARIABILITY  OF  PRODUCT  A  EXCEED  THAT  OF  PRODUCT  B? 

We  wish  to  test  whether  the  variability  in  performance  of  product  A  exceeds  that  of  product  B. 
We  wish,  as  a  result  of  analysis  of  the  data,  to  make  one  of  the  following  decisions: 

(a)  The  variability  of  product  A  exceeds  that  of  product  B. 

(b)  There  is  no  reason  to  believe  that  the  variability  of  product  A  exceeds  the  variability  of 
product  B. 
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In  terms  of  Data  Sample  4-2.1,  let  us  suppose  that — in  advance  of  looking  at  the  data! — the 
important  question  is:  Does  the  variability  of  Method  A  exceed  that  of  Method  B? 


Procedure 

(1)  Choose  a,  the  level  of  significance  of  the 
test. 

(2)  Look  up  for  n A  —  1,  nB  -  1  degrees  of 

freedom,  in  Table  A-5. 

(3)  Compute  Sx,  ss,  the  sample  variances  of 
the  observations  from  A  and  B,  respec¬ 
tively. 

(4)  Compute  F  =  Sx/s5fl. 

(5)  If  F  >  Fy~a,  decide  that  the  variability  of 
product  A  exceeds  that  of  B;  otherwise, 
there  is  no  reason  to  believe  that  the  vari¬ 
ability  of  A  is  greater  than  that  of  B. 

(6)  Note  that  the  interval  above 

_ i _ (SA\ 

Fi^  (nA  -  1,  Tie  -  1)  \sy 

is  a  100  (1  —  a)  %  confidence  interval  esti¬ 
mate  Of  Vx/crfi. 


Example 

(1)  Let  a  =  .05 

(2)  «x  -  1  «  11 

nB  -  1  =  11 

F.,s(  11,  11)  =  2.82 

(3)  s2x  =  5545 

ssfl  =  4073 

(4)  F  =  1.36 

(5)  Since  1.36  is  not  larger  than  2.82,  there  is 
no  reason  to  believe  that  the  variability  of 
Method  A  is  greater  than  the  variability 
of  Method  B. 

(6) 

F.n(  11,  fl)  “  0,35 

The  interval  above  0.48  (i.e.,  0.35  X  1.36) 
is  a  95%  confidence  interval  estimate  of  the 
ratio  of  the  true  variances,  oxAs- 


Operating  Characteristics  of  the  Test.  Figures  4-3,  4-4,  and  4-5  provide  operating-characteristic 
(OC)  curves  of  the  test  for  a  =  0.05  and  various  combinations  of  nA  and  nB.  Let  <rA  and  aB  denote 
the  true  standard  deviations  of  the  products  A  and  B,  respectively.  These  OC  curves  show  the 
probability  0  =  0  (X|.05,  n)  of  failing  to  conclude  that  <j A  exceeds  cB  when  <rA  =  \aB  with  X  >  1 
and  the  test  is  carried  out  at  the  a  =  0.05  levei  of  significance  using  the  values  of  Sx  and  sB  derived 
from  samples  of  size  nA  and  nB,  respectively.  Similar  OC  curves  for  the  case  of  a  =s  0.01  and 
r?x  =  nB  are  given  in  Figure  7.4  of  Bowker  and  Lieberman,<2)  OC  curves  are  easily  constructed  for 
other  values  of  nA  and  nB  —  and,  if  desired,  other  values  of  a  —  by  utilizing  the  fact  that  if  the  test  is 
conducted  at  the  a  level  of  significance  using  values  of  sA  and  sB  based  on  samples  of  size  nA  and  nB, 
respectively,  then  the  probability  of  failing  to  conclude  that  aA  exceeds  cB  when  oA  =  X<rB  is  exactly 
0  for 


X  =  X  («,  ft  »x,  nB)  =  J  F'~^inA  - 

\  Ft s  (»x  -  1,  nB  - 


■  1) 
1) 


~  VFi_«  (nA  -  1,  nB  -  1)  •  Fi_„  ( nB  -  1,  nA  -  1) 


where  Fp  (n u  n2)  is  the  P-probability  level  of  F  for  n,  and  n2  degrees  of  freedom,  as  given  in 
Table  A-5.  Values  of  <f>  (a,  ft  n,,  n2)  =  X2  {a,  ft  nA,  nB)  corresponding  to  o  =  0.05  and  a  =  0.01, 
for  ft  =  0.005,  0.01,  0.025,  0.05,  0.10,  0.25,  0.50,  0.75,  0.90,  0.95,  0.975,  0.99,  and  0.995  are  given  in 
Tables  8.3  and  8.4  of  Eisenhart<3)  for  all  combinations  of  values  of  ni  =  nA  —  1  and  7ia  =  n„  —  1 
derivable  from  the  sequence  1(1)30(10)100,  120,  ». 
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Figure  U-S.  Operating  characteristics  of  the  one-sided  F-test  to  determine  whether  the  standard  deviation 
aA  of  product  A  exceeds  the  standard  deviation  aB  of  product  B  (a  =  .05;  nA  =  nB). 


Adapted  with  permission  from  Anna!*  of  Mathematical  Statistic i,  Vol.  17,  No.  Z,  June  1946,  from  article  entitled  "Operating  Characteristic  for  the 
Common  Statistical  Testa  of  Significance  by  C.  D.  Ferris,  F.  £.  Grubbs,  and  C.  L.  Weaver. 


Selection  of  Sample  Size.  If  we  choose 
nA  ~  nB  =  n 


a,  the  significance  level  of  the  test- 

13,  the  probability  of  failing  to  detect  that  aA  exceeds  <rB  when  <rA  =  X<rs 

then  for  a  =  0.05,  we  may  use  the  OC  curve  of  Figure  4-3  to  determine  the  necessary  common 
sample  size  n. 
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Figure  4- 4.  Operating  characteristics  of  the  one-sided  F-test  to  determine  whether  the  standard  deviation 
a A  of  product  A  exceeds  the  standard  deviation  oB  of  product  B 
(a  =  .05;  nA  =  nB,  SnA  =  ZnB,  ZnA  *  nB). 

Adapted  with  permiaaion  from  innaft  of  Mathematical  Statute*.  Vol.  17,  No.  2,  Juno  194ft.  from  artid*  ontltlod  "Operating  Charnctorlatloa  for  tha 
Common  Statistical  TetU  of  Significance'  by  C.  D.  Ferria,  F.  E.  Grubb*,  and  C.  L  Weaver. 


Example:  Choose 

*  -  Y»  -  !-5 

a  =  0.05 
ft  -  0.05 

then  from  Figure  4-3  it  is  seen  that  n  =  50  is  too  small  and  n  =  75  a  bit  too  large.  Visual  inter¬ 
polation  suggests  n  =  70. 
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Figure  4-5.  Operating  characteristics  of  the  one-sided  F-test  to  determine  whether  the  standard  deviation 
<j,t  of  product  A  exceeds  the  standard  deviation  aH  of  product  B 
(a  -  .05;  nA  =  nH,  2nA  *  SnH,  nA  «=  2nH). 

Adapted  with  permlaeton  f»-om  Anna!*  of  Maihrmatirai  SlafiriuM,  Vol.  17,  No.  2,  June  1946,  from  article  entitled  '’Operating  Characteristics  for  the 
Common  Statistical  Testa  of  Significance"  by  C.  D.  Ferria,  F,  K,  Gruhta,  and  0.  L.  Weaver- 


Alternatively,  for  nA  =  n„  =  n  one  may  compute  the  necessary  sample  size  from  the  approximate 
formula 


n  =  n  (a,  i S,  A)  =  2  + 


/  z  i_„  +  Zi-jV 
\  log,x  / 


where  zr  is  the  P-probability  point  of  the  standard  normal  variable  z,  values  of  which  are  given  in 
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Table  A-2  ,for  various  values  of  P.  Thus  in  the  foregoing  example  we  find 


n 


n  ,  /T.645  +  1.645V 
£  +  \  .4055  ) 

2  +  (Mi)'  • 2  +  <8-U)’  “1  2 3 * *  + 66-8 

68. 


If  instead  of  choosing  nA  =  nB  we  choose  3 nA  -  2 nB  or  2 nA  =  nB,  then  for  a  =  0.05  we  may 
use  the  OC  curves  of  Figure  4-4  to  determine  the  necessary  combination  of  sample  sizes  nA  and  nB- 
Similarly,  Figure  4-5  may  be  used  if  it  is  desired  to  have  2 nA  =  3 nB  or  nA  =  2nB.  Alternatively, 
one  may  evaluate  the  harmonic  mean  h  of  nA  —  2  and  nB  —  2  from  the  approximate  formula 


h  = 


/ Zl —g  ^1— " 

\  logA  ) 


and  then  determine  the  integer  values  of  nA  and  nB  (satisfying  any  additional  requirements,  e.g., 
nA  =  2 wB)  that  most  closely  satisfy  the  equation 


1  =  1  i—1—  + _ 1 — l 

h  2  \nA  —  2  nB  —  2f 
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CHAPTER  5 

CHARACTERIZING  LINEAR  RELATIONSHIPS 
BETWEEN  TWO  VARIABLES 


5-1  INTRODUCTION 


In  many  situations  it  is  desirable  to  know 
something  about  the  relationships  between  two 
characteristics  of  a  material,  product,  or  proc¬ 
ess.  In  some  cases,  it  may  be  known  from 
theoretical  considerations  that  two  properties 
are  functionally  related,  and  the  problem  is  to 
find  out  more  about  the  structure  of  this  rela¬ 
tionship.  In  other  cases,  there  is  interest  in 
investigating  whether  there  exists  a  degree  of 
association  between  two  properties  which  could 
be  used  to  advantage.  For  example,  in  specify¬ 
ing  methods  of  test  for  a  material,  there  may  be 
two  tests  available,  both  of  which  reflect  per¬ 
formance,  but  one  of  which  is  cheaper,  simpler, 
or  quicker  to  run.  If  a  high  degree  of  associa¬ 
tion  exists  between  the  two  tests,  we  might  wish 
to  run  regularly  only  the  simpler  test. 


In  this  chapter,  we  deal  only  with  linear  rela¬ 
tionships.  Curvilinear  relationships  are  dis¬ 
cussed  in  Chapter  6  (see  Paragraph  6-5).  It  is 
worth  noting  that  many  nonlinear  relationships 
may  be  expressed  in  linear  form  by  a  suitable 
transformation  (change  of  variable).  For  exam¬ 
ple,  if  the  relationship  is  of  the  form  Y  =  aX \ 

then  log  Y  =  log  a  +  b  log  X.  Putting  Yr  ~ 

log  y,  bB  —  log  a,  b i  =  b,  XT  =  log  X,  we  have 

the  linear  expression  YT  =  b0  +  b iXr  in  terms 

of  the  new  (transformed)  variables  XT  and  Yt- 

A  number  of  common  linearizing  transforma¬ 
tions  are  summarized  in  Table  5-4  and  are  dis¬ 
cussed  in  Paragraph  5-4.4. 


5-2  PLOTTING  THE  DATA 


Where  only  two  characteristics  are  involved, 
the  natural  first  step  in  handling  the  experi¬ 
mental  results  is  to  plot  the  points  on  graph 
paper.  Conventionally,  the  independent  vari¬ 
able  X  is  plotted  on  the  horizontal  scale,  and  the 
dependent  variable  Y  is  plotted  on  the  vertical 
scale. 

There  is  no  substitute  for  a  plot  of  the  data  to 
give  some  idea  of  the  general  spread  and  shape 
of  the  results.  A  pictorial  indication  of  the 
probable  form  and  sharpness  of  the  relation¬ 
ship,  if  any,  is  indispensable  and  sometimes  may 
save  needless  computing.  When  investigating 


a  structural  relationship,  the  plotted  data  will 
show  whether  a  hypothetical  linear  relationship 
is  borne  out;  if  not,  we  must  consider  whether 
there  is  any  theoretical  basis  for  fitting  a  curve 
of  higher  degree.  When  looking  for  an  empiri¬ 
cal  association  of  two  characteristics,  a  glance  at 
the  plot  will  reveal  whether  such  association  is 
likely  or  whether  there  is  only  a  patternless 
scatter  of  points. 

In  some  cases,  a  plot  will  reveal  unsuspected 
difficulties  in  the  experimental  setup  which 
must  be  ironed  out  before  fitting  any  kind  of 
relationship.  An  example  of  this  occurred  in 
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measuring  the  time  required  for  a  drop  of  dye  to 
travel  between  marked  distances  along  a  water 
channel.  The  channel  was  marked  with  dis¬ 
tance  markers  spaced  at  equal  distances,  and  an 
observer  recorded  the  time  at  which  the  dye 
passed  each  marker.  The  device  used  for  re¬ 
cording  time  consisted  of  two  clocks  hooked  up 
so  that  when  one  was  stopped,  the  other  started: 
Clock  1  recorded  the  times  for  Distance  Mark¬ 
ers  1,  3,  5,  etc. ;  and  Clock  2  recorded  times  for 
the  even-numbered  distance  markers.  When 
the  elapsed  times  were  plotted,  they  looked 
somewhat  as  shown  in  Figure  5-1.  It  is  ob¬ 
vious  that  there  was  a  systematic  time  differ¬ 
ence  between  odd  and  even  markers  (presuma¬ 
bly  a  lag  in  the  circuit  connecting  the  two 
clocks).  One  could  easily  have  fitted  a  straight 
line  to  the  odd-numbered  distances  and  a  dif¬ 
ferent  line  to  the  even-numbered  distances,  with 
approximately  constant  difference  between  the 
two  lines.  The  effect  was  so  consistent,  how¬ 


ever,  that  the  experimenter  quite  properly 
decided  to  find  a  better  means  of  recording 
travel  times  before  fitting  any  line  at  all. 

If  no  obvious  difficulties  are  revealed  by  the 
plot,  and  the  relationship  appears  to  be  linear, 
then  a  line  Y  =  60  +  biX  ordinarily  should  be 
fitted  to  the  data,  according  to  the  procedures 
given  in  this  Chapter.  Fitting  by  eye  usually  is 
inadequate  for  the  following  reasons: 

(a)  No  two  people  would  fit  exactly  the  same 
line,  and,  therefore,  the  procedure  is  not  ob¬ 
jective; 

(b)  We  always  need  some  measure  of  how 
well  the  line  does  fit  the  data,  and  of  the  uncer¬ 
tainties  inherent  in  the  fitted  line  as  a  repre¬ 
sentation  of  the  true  underlying  relationship — 
and  these  can  be  obtained  only  when  a  formal, 
well-defined  mathematical  procedure  of  fitting 
is  employed. 
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Figure  5-1.  Time  required  for  a  drop  of  dye  to  travel 
between  distance  markers. 
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5-3  TWO  IMPORTANT  SYSTEMS  OF  LINEAR  RELATIONSHIPS 


Before  giving  the  detailed  procedure  for  fit¬ 
ting  a  straight  line,  we  discuss  different  physical 
situations  which  can  be  described  by  a  linear 
relationship  between  two  variables.  The  meth¬ 
ods  of  description  and  prediction  may  be  differ¬ 
ent,  depending  upon  the  underlying  system. 
In  general,  we  recognize  two  different  and  im¬ 
portant  systems  which  we  call  Statistical  and 
Functional.  It  is  not  possible  to  decide  which  is 
the  appropriate  system  from  looking  at  the 
data.  The  distinction  must  be  made  before 
fitting  the  line — indeed,  before  taking  the 
measurements. 

5-3.1  FUNCTIONAL  RELATIONSHIPS 

In  the  case  of  a  Functional  Relationship, 
there  exists  an  exact  mathematical  formula  {y  as 
a  function  of  x)  relating  the  two  variables,  and 
the  only  reason  that  the  observations  do  not  fit 
this  equation  exactly  is  because  of  disturbances 
or  errors  of  measurement  in  the  observed  values 
of  one  or  both  variables.  We  discuss  two  cases 
of  this  type: 

FI — Errors  of  measurement  affect  only  one 
variable  (7).  (See  Fig.  5-2). 

FII — Both  variables  ( X  and  7)  are  subject  to 
errors  of  measurement.  (See  Fig.  5-3). 

Common  situations  that  may  be  described  by 
Functional  Relationships  include  calibration 
lines,  comparisons  of  analytical  procedures,  and 
relationships  in  which  time  is  the  X  variable. 

For  instance,  we  may  regard  Figure  5-2  as 
portraying  the  calibration  of  a  straight-faced 
spring  balance  in  terms  of  a  series  of  weights 
whose  masses  are  accurately  known.  By 
Hooke’s  Law,  the  extension  of  the  spring,  and 
hence  the  position  y  of  the  scale  pointer,  should 
be  determined  exactly  by  the  mass  x  upon  the 
pan  through  a  linear  functional  relationship* 
y  =  /30  +  0y  x.  In  practice,  however,  if  a  weight 


*  Note  on  Notation  for  Functional  Relationships: 

We  have  used  x  and  y  to  denote  the  true  or  accurately 
known  values  of  the  variables,  and  A'  and  Y  to  denote 
their  values  measured  with  error.  In  the  FI  Relation¬ 
ship,  the  independent  variable  is  always  without  error, 
and  therefore  in  our  discussions  of  the  FI  case  and  in  the 
paragraph  headings  we  always  use  x.  In  the  Worksheet, 


of  mass  Xi  is  placed  upon  the  pan  repeatedly  and 
the  position  of  the  pointer  is  read  in  each 
instance,  it  usually  is  found  that  the  readings  Yy 
are  not  identical,  due  to  variations  in  the  per¬ 
formance  of  the  spring  and  to  reading  errors. 
Thus,  corresponding  t.o  the  mass  x,  there  is  a 
distribution  of  pointer  readings  7,;  correspond¬ 
ing  to  mass  x? ,  a  distribution  of  pointer  readings 
7j;  and  so  forth — as  indicated  in  Figure  5-2. 
It  is  customary  to  assume  that  these  distribu¬ 
tions  are  normal  (or,  at  least  symmetrical  and 
all  of  the  same  form)  and  that  the  mean  of  the 
distribution  of  7/s  coincides  with  the  true  value 
Vi  =  Po  +  Pi  Xi . 

If,  instead  of  calibrating  the  spring  balance  in 
terms  of  a  series  of  accurately  known  weights, 
we  were  to  calibrate  it  in  terms  of  another 
spring  balance  by  recording  the  corresponding 
pointer  positions  when  a  series  of  weights  are 
placed  first  on  the  pan  of  one  balance  and  then 
on  the  pan  of  the  other,  the  resulting  readings 
( X  and  7)  would  be  related  by  a  linear  struc¬ 
tural  relationship  FII,  as  shown  in  Figure  5-3, 
inasmuch  as  both  X  and  7  are  affected  by  errors 
of  measurement.  In  this  case,  corresponding 
to  the  repeated  weighings  of  a  single  weight  wy 
(whose  true  mass  need  not  be  known),  there  is  a 
joint  distribution  of  the  pointer  readings 
(X!  and  7i)  on  the  two  balances,  represented  by 
the  little  transparent  mountain  centered  over 
the  true  point  (xb  yx)  in  Figure  5-3;  similarly  at 
points  (x2,  y2)  and  (x3,  y3),  corresponding  to  re¬ 
peated  weighings  of  other  weights  w2  and  w3, 
respectively.  Finally,  it  should  be  noticed  that 
this  FII  model  is  more  general  than  the  FI 
model  in  that  it  does  not  require  linearity  of 
response  of  each  instrument  to  the  independent 
variable  w,  but  merely  that  the  response  curves 

and  Procedures  and  Examples  lor  the  FI  case,  however, 
we  use  X  and  Y  because  of  the  computational  similarity 
to  other  caseB  discussed  in  this  Chapter  (i.e.,  the  computa¬ 
tions  lor  the  Statistical  Relationships). 

In  the  FII  case,  both  variables  are  subject  to  error,  and 
clearly  we  use  X  and  Y  everywhere  for  the  observed 
values. 
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of  the  two  instruments  be  linearly  related,  that 
is,  that  X  =  a  +  b  •  f(w)  and  Y  =  c  4-  d  •  f(w), 
where  f(w)  may  be  linear,  quadratic,  exponen¬ 
tial,  logarithmic,  or  whatever. 

Table  5-1  provides  a  concise  characterization 


of  FI  and  FI  I  relationships.  Detailed  prob¬ 
lems  and  procedures  with  numerical  examples 
for  FI  relationships  are  given  in  Paragraphs 
5-4.1  and  5-4.2,  and  for  FII  relationships  in 
Paragraph  5-4.3. 


Y 
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Figure  5-3.  Linear  functional  relationship  of  Type  FII 
(both  X  and  Y  affected  by  measurement  errors). 


5-3.2  STATISTICAL  RELATIONSHIPS 

In  the  case  of  a  Statistical  Relationship,  there 
is  no  exact  mathematical  relationship  between 
X  and  Y ;  there  is  only  a  statistical  association 
between  the  two  variables  as  characteristics  of 
individual  items  from  some  particular  popula¬ 
tion.  If  this  statistical  association  is  of  bi¬ 
variate  normal  type  as  shown  in  Figure  5-4, 
then  the  average  value  of  the  Y’s  associated  with 
a  particular  value  of  X,  say  Yx,  is  found  to  de¬ 
pend  linearly  on  X,  i.e.,  Yx  =  0o  +  $i  X;  simi¬ 
larly,  the  average  value  of  the  X’s  associated 
with  a  particular  value  of  Y,  say  Xy,  depends 
linearly  on  Y  (Fig.  5-4)  i.e.,  XY  =  Y; 


but — and  this  is  important! — the  two  lines  are 

1  8 

not  the  same,  i.e.,  ^  ~  and  * 

Pi  Pi 


*  Strictly,  we  should  write 
%.r  *■  Pd  +  0i  X  , 
and 

mx-r  =  Ph  +  V 

to  conform  to  our  notation  of  using  m  to  signify  a 
population  mean.  But  this  more  exact  notation  tends 
to  conceal  the  parallelism  of  the  curve-fitting  processes 
in  the  FI  and  SI  situations.  Consequently,  to  preserve 
appearances  hye  and  in  the  sequel,  we  use  Yx  in  place 
of  mr.x  and  Xr  in  place  of  and  it  should  be 

remembered  that  these  signify  population  meant. 
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Figure  5-k  ■  A  normal  bivariate  frequency  surface. 

If  a  random  sample  of  items  is  drawn  from  the 
population,  and  the  two  characteristics  X  and  Y 
are  measured  on  each  item,  then  typically  it  is 
found  that  errors  of  measurement  are  negligible 
in  comparison  with  the  variation  of  each  char¬ 
acteristic  over  the  individual  items.  This 
general  case  is  designated  SI.  A  special  case 
(involving  preselection  or  restriction  of  the 
range  of  one  of  the  variables)  is  denoted  by  SII. 

SI  Relationship*.  In  this  case,  a  random 
sample  of  items  is  drawn  from  some  definite 
population  (material,  product,  process,  or 
people),  and  two  characteristics  are  measured 
on  each  item. 

A  classic  example  of  this  type  is  the  relation¬ 
ship  between  height  and  weight  of  men.  Any 
observant  person  knows  that  weight  tends  to 
vary  with  height,  but  also  that  individuals  of 
the  same  height  may  vary  widely  in  weight.  It 
is  obvious  that  the  errors  made  in  measuring 
height  or  weight  are  very  small  compared  to 
this  inherent  variation  between  individuals. 
We  surely  would  not  expect  to  predict  the  exact 


weight  of  one  individual  from  his  height,  but  we 
might  expect  to  be  able  to  estimate  the  average 
weight  of  all  individuals  of  a  given  height. 

The  height-weight  example  is  given  as  one 
which  is  universally  familiar.  Such  examples 
also  exist  in  the  physical  and  engineering  sci¬ 
ences,  particularly  in  cases  involving  the  inter¬ 
relation  of  two  test  methods.  In  many  cases 
there  may  be  two  tests  that,  strictly  speaking, 
measure  two  basically  different  properties  of  a 
material,  product,  or  process,  but  these  proper¬ 
ties  are  statistically  related  to  each  other  in 
some  complicated  way  and  both  are  related  to 
some  performance  characteristic  of  particular 
interest,  one  usually  more  directly  than  the 
other.  Their  interrelationship  may  be  ob¬ 
scured  by  inherent  variations  among  sample 
units  (due  to  varying  density,  for  example). 
We  would  be  very  interested  in  knowing 
whether  the  relationship  between  the  two  is 
sufficient  to  enable  us  to  predict  with  reasonable 
accuracy,  from 'a  value  given  by  one  test,  the 
average  value  to  be  expected  for  the  other — 
particularly  if  one  test  is  considerably  simpler 
or  cheaper  than  the  other. 

The  choice  of  which  variable  to  call  X  and 
which  variable  to  call  Y  is  arbitrary — actually 
there  are  two  regression  lines.  If  a  statistical 
association  is  found,  ordinarily  the  variable 
which  is  easier  to  measure  is  called  X.  Note 
well  that  this  is  the  only  case  of  linear  relation¬ 
ship  in  which  it' may  be  appropriate  to  fit  two 
different  lines,  one  for  predicting  Y  from  X  and 
a  different  one  for  predicting  X  from  Y,  and  the 
only  case  in  which  the  sample  correlation  co¬ 
efficient  r  is  meaningful  as  an  estimate  of  the 
degree  of  association  of  X  and  Y  in  the  popula¬ 
tion  as  measured  by  the  population  coefficient 
of  correlation  p  =*  The  six  sets  of  con¬ 

tour  ellipses  shown  in  Figure  5-5  indicate  the 
manner  in  which  the  location,  shape,  and  orien¬ 
tation  of  the  normal  bivariate  distribution 
varies  with  changes  of  the  population  means 
(m.v  and  my)  and  standard  deviations  (errand  <?r) 
of  X  and  Y  and  their  coefficient  of  correlation  in 
the  population  (pVK). 

If  p  =  =tl,  all  the  points  lie  on  a  line  and 
Y  —  Po  +  PiX  and  X  =  /^  +  £)[Y  coincide. 
If  p  =  +1,  the  slope  is  positive,  and  if  p  =  —  1, 
the  slope  is  negative.  If  p  =  0,  then  X  and  Y 
are  said  to  be  uncorrelated. 
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Figure  5-5.  Contour  ellipses  for  normal  bivariate  distributions 
having  different  values  of  the  five  parameters  mx,  mY,  ax,  aY,  pXY. 

Adapted  with  permiarfon  from  SUitinticGl  Inftrtncn  by  Helen  M.  Walker  and  Joseph  Lev,  copy¬ 
right,  1953,  Holt,  Rinehart  and  Wimton,  Inc.,  NewYork,  N.  Y. 


SI1  Relation* hip*.  The  general  case  described 
above  (SI)  is  the  most  familiar  example  of  a 
statistical  relationship,  but  we  also  need  to  con¬ 
sider  a  common  case  of  Statistical  Relationship 
(SII)  that  must  be  treated  a  bit  differently.  In 
SII,  one  of  the  two  variables,  although  a  ran¬ 
dom  variable  in  the  population,  is  sampled  only 
within  a  limited  range  (or  at  selected  preas¬ 
signed  values).  In  the  height-weight  example, 
suppose  that  the  group  of  men  included  only 


those  whose  heights  were  between  5'4"  and 
5'8\  We  now  are  able  to  fit  a  line  predicting 
weight  from  height,  but  are  unable  to  determine 
the  correct  line  for  predicting  height  from 
weight.  A  correlation  coefficient  computed 
from  such  data  is  not  a  measure  of  the  true 
correlation  among  height  and  weight  in  the  (un¬ 
restricted)  population. 

The  restriction  of  the  range  of  X,  when  it  is 
considered  as  the  independent  variable,  does 
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not  spoil  the  estimates  of  Yx  when  we  fit  the 
line  Yx  =  b0  +  btX.  The  restriction  of  the 
range  of  the  dependent  variable  (i.e.,  of  Y  in 
fitting  the  foregoing  line,  or  of  X  in  fitting  the 
line  XY  =  b'0  +  b[Y),  however,  gives  a  seriously- 
distorted  estimate  of  the  true  relationship. 
This  is  evident  from  Figure  5-6,  in  which  the 
contour  ellipses  of  the  top  diagram  serve  to 
represent  the  bivariate  distribution  of  X  and  Y 
in  the  unrestricted  population,  and  the  “true” 
regression  lines  of  Yx  on  X  and  Xy  on  Y  are 
indicated.  The  central  diagram  portrays  the 
situation  when  consideration  is  restricted  to 
items  in  the  population  for  which  a  <  X  <  b. 
It  is  clear  that  for  any  particular  X  in  this  in¬ 
terval,  the  distribution  and  hence  the  mean 
Yx  of  the  corresponding  Y’s  is  the  same  as  in 
the  unrestricted  case  (top  diagram).  Conse¬ 
quently,  a  line  of  the  form  Yx  =  b0  +  biX  fitted 
to  data  involving  either  a  random  or  selected  set 
of  values  of  X  between  X  -  a  and  X  =  b,  but 
with  no  selection  or  restrictions  on  the  corre¬ 
sponding  Y’s,  will  furnish  an  unbiased  estimate 
of  the  true  regression  line  Yx  =  6  a  +  61X  in  the 
population  at  large.  In  contrast,  if  considera¬ 
tion  is  restricted  to  items  for  which  c  <  Y  <  d, 
as  indicated  in  the  bottom  diagram^  then  it  is 
clear  that  the  mean  value,  say  Y'x,  of  the 
(restricted)  Y’s  associated  with  any  particular 
value  of  X  >  mx  will  be  less  than  the  corre¬ 
sponding  mean  value  Yx  in  the  population  as 
a  whole.  Likewise,  if  X  <  mx,  then  the  mean 
)”Y  of  the  corresponding  (restricted)  Y’s  will 
be  greater  than  Y.v  in  the  population  as  a  whole. 
Consequently,  a  line  of  the  form  Y'x  -  ba  -f  b tX 
fitted  to  data  involving  selection  or  restriction 
of  Y’s  will  not  furnish  an  unbiased  estimate  of 
the  true  regression  line  Yv  =  /3„  +  $\X  in  the 
population  as  a  whole,  and  the  distortion  may 
be  serious.  In  other  words,  introducing  a  re¬ 
striction  with  regard  to  A"  does  not  bias  infer¬ 
ences  with  regard  to  Y,  when  Y  is  considered  as 
the  dependent  variable,  but.  restricting  Y  will 
distort  the  dependence  of  Y x  on  X  so  that  the 
relationship  observed  will  not  be  representative 
of  the  true  underlying  relationship  in  the  popu¬ 
lation  as  a  whole.  Obviously,  there  is  an 
equivalent  statement  in  which  the  roles  of  X 
and  Y  are  reversed.  For  further  discussion  and 
illustration  of  this  point,  and  of  the  correspond¬ 
ing  distortion  of  the  sample  correlation  coeffi- 


Figure  5-6.  Diagram  showing  effect  of  restrictions 
of  X  or  Y  on  the  regression  of  Y  on  X. 


cient  r  as  a  measure  of  the  true  coefficient  of 
correlation  p  in  the  populations,  when  either  X 
or  Y  is  restricted,  see  Eisenhart0’  and  Ezekiel. (!) 

As  an  engineering  example  of  SII,  consider  a 
study  of  watches  to  investigate  whether  there 
was  a  relationship  between  the  cost  of  a  stop 
watch  and  its  temperature  coefficient.  It  was 
suggested  that  a  correlation  coefficient  be  com¬ 
puted.  This  was  not  possible  because  the 
watches  had  not  been  selected  at  random  from 
the  total  watch  production,  but  a  deliberate 
effort  had  been  made  to  obtain  a  fixed  number 
of  low-priced,  medium-priced,  and  high-priced 
stop  watches. 

In  any  given  case,  consider  carefully  whether 
one  is  measuring  samples  as  they  come  (and 
thereby  accepting  the  values  of  both  properties 
that  come  with  the  sample)  which  is  an  SI  Rela¬ 
tionship,  or  whether  one  selects  samples  which 
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are  known  to  have  a  limited  range  of  values  of  X 
(which  is  an  SII  Relationship). 

Table  5-1  gives  a  brief  summary  characteriza¬ 
tion  of  SI  and  SII  Relationships.  Detailed 


problems  and  procedures  with  numerical  exam¬ 
ples  are  given  for  SI  relationships  in  Paragraph 
5-5.1  and  for  SII  relationships  in  Paragraph 
5-5.2. 


BASIC  WORKSHEET  FOR  ALL  TYPES  OF  LINEAR  RELATIONSHIPS 


X  denotes 


Y  denotes 


2X  =  _ 

X  =_ 


sr  =  _ 

y  = ... 


Number  of  points:  n  = 


Step  (1)  ZXY 

(2)  (SX)  (SY)/n  = 


(3) 

Sz„ 

=  Step  (1)  —  Step 

(2) 

(4) 

2X* 

— 

(7) 

2Y* 

= 

(5) 

(zxy/n 

= . 

— 

— 

(8) 

(ZYY/n 

= 

(6) 

= 

Step  (4) 

-  Step  (5) 

(9) 

svv 

= 

(10) 

b 

0 1—0 

XX 

= 

Step  (3) 

4-  Step  (6) 

(14) 

{sivy 

s„ 

= 

(ID 

Y 

= . 

— 

(15) 

(n  —  2)  slr 

= 

(12) 

biX 

= . 

— 

- - — ..  - . . 

(16) 

«v 

= 

(13) 

bn  =  y  - 

b,X  = 

Step  (11) 

—  Step  (12) 

Sy 

= 

Step  (7 )  -  Step  (8) 


Step  (9)  -  Step  (14) 
Step  (15)  4-  (w  —  2) 


Equation  of  the  line: 
Y  =  bo  +  WX 


sh  ~ - 

s,0  = - 


Estimated  variance  of  the  slope: 


si  =  &  =  _Step  (16) 

Estimated  variance  of  intercept: 


si»  — 


Step  (6) 


Note:  The  following  are  algebraically  identical: 

S„  =  Z(X  -  X)1;  Svv  =  2(7  -  Y)s;  SIU  =  2(X  -  X)  (Y  -  Y). 


Ordinarily,  in  hand  computation,  it  is  preferable  to  compute  as  shown  in  the  steps  above.  Carry 
all  decimal  places  obtainable  -  i.e.,  if  data  are  recorded  to  two  decimal  places,  carry  four  places  in 
Steps  (1)  through  (9)  in  order  to  avoid  losing  significant  figures  in  subtraction. 
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5-4  PROBLEMS  AND  PROCEDURES 

5-4.!  FI  RELATIONSHIPS  (General  Case) 

There  is  an  underlying  mathematical  (func¬ 
tional)  relationship  between  the  two  variables, 
of  the  form  y  -  1 30  +  /3i*.  The  variable  x  can 
be  measured  relatively  accurately.  Measure¬ 
ments  Y  of  the  value  of  y  corresponding  to  a 
given  x  follow  a  normal  distribution  with  mean 
0o  +  A*  and  variance  a\.x  which  is  independent 
of  the  value  of  x.  Furthermore,  we  shall  as¬ 
sume  that  the  deviations  or  errors  of  a  series  of 
observed  Y’s,  corresponding  to  the  same  or  dif¬ 
ferent  z’s,  all  are  mutually  independent.  See 
Paragraph  5-3.1  and  Table  5-1. 

The  general  case  is  discussed  here,  and  the 
special  case  where  it  is  known  that  0D  =  0  (i.e., 
a  line  known  to  pass  through  the  origin)  is  dis¬ 
cussed  in  Paragraph  5-4.2.  The  procedure  dis¬ 
cussed  here  also  will  be  valid  if  in  fact  0O  -  0 
even  though  this  fact  is  not  known  beforehand. 
However,  when  it  is  known  that  0O  =  0,  the  pro¬ 
cedures  of  Paragraph  5-4.2  should  be  followed 
because  they  are  simpler  and  somewhat  more 
efficient. 

It  will  be  noted  that  SII,  Paragraph  5-5.2,  is 
handled  computationally  in  exactly  the  same 
manner  as  FI,  but  both  the  underlying  assump¬ 
tions  and  the  interpretation  of  the  end  results 
are  different. 


FOR  FUNCTIONAL  RELATIONSHIPS 


Data  Sample  5-4.1 — Young's  Modulus  vs. 

Temperature  for  Sapphire  Rods 

Observed  values  ( Y )  of  Young's  modulus  <y) 
for  sapphire  rods  measured  at  different  tempera¬ 
tures  (x)  are  given  in  the  following  table.  There 
is  assumed  to  be  a  linear  functional  relationship 
between  the  two  variables  x  and  y.  (For  the 
purpose  of  computation,  the  observed  Y  values 
were  coded  by  subtracting  4000  from  each.  To 
express  the  line  in  terms  of  the  original  units, 
add  4000  to  the  computed  intercept;  the  slope 
will  not  be  affected.)  The  observed  data  are 
plotted  in  Fig’  .re  5-7. 


X 

-  Temperature 
°C 

y 

-  Young’s 
Modulus 

Coded  Y 
=  Young’s 
Modulus 
minus  4000 

30 

4642 

642 

100 

4612 

612 

200 

4565 

565 

300 

4513 

513 

400 

4476 

476 

500 

4433 

433 

600 

4389 

389 

700 

4347 

347 

800 

4303 

303 

900 

4251 

251 

1000 

4201 

201 

1100 

4140 

140 

1200 

4100 

100 

1300 

4073 

73 

1400 

4024 

24 

1500 

3999 

-1 
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Figure  5-7.  Young’s  modulus  of  sapphire  rods  as  a  function 
of  temperature — an  FI  relationship. 


5-4. 1.1  What  It  th«  B«st  Lin*  to  b«  Ut«d  for 
Estimating  y  From  Givon  Valuot  of  xV 

CAUTION:  Extrapolation,  i.e.,  use  of  the 
line  for  prediction  outside  the  range  of  data 
from  which  the  line  was  computed,  may 
lead  to  highly  erroneous  conclusions. 

Procedure 

Using  Worksheet  (See  Worksheet  5-4.1), 
compute  the  line  Y  =  b0  -f  b,x.  This  is  an 
estimate  of  the  true  equation  y  =  -f  fra:. 
The  method  of  fitting  a  line  given  here  is  a 


particular  application  of  the  general  method  of 
least  squares.  From  Data  Sample  5-4.1,  the 
equation  of  the  fitted  line  (in  original  units)  is: 

Y  =  4654.9846  -  0.44985482  a;. 

The  equation  in  original  units  is  obtained  by 
adding  4000  to  the  computed  intercept  bo. 
Since  the  F's  were  coded  by  subtracting  a  con¬ 
stant,  the  computed  slope  bt  was  not  affected. 
In  Figure  5-8,  the  line  is  drawn  and  confidence 
limits  for  the  line  (computed  as  described  in 
Paragraph  5-4.1.2.1)  also  are  shown. 
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WORKSHEET  5*4.1 
EXAMPLE  OF  FI  RELATIONSHIP 
YOUNG'S  MODULUS  AS  FUNCTION  OF  TEMPERATURE 


X  denotes 

Temperature,  °C 

Y  denotes 

Young's  Modulus  —  4000 

ZX  = 

12030 

ZY  = 

5068 

X  = 

751.875 

Y  = 

316.75 

Number  of  points:  n  =  16 


(1)  XXY 

(2)  (XX)  (ZY)/n  = 

(3)  S„ 


(4) 

ZX* 

12,400,900 

(5) 

N 

s' 

II 

9,045,056.25 

(6) 

s„ 

3,355,843.75 

(10) 

b  —  — -  — 

bl  ~  s„ 

-.449,854,82 

(ID 

Y 

316.75 

(12) 

btX 

-338.2346 

(13) 

bo  =  Y  -  biX  = 

_ 654.9846 

b0  (in  original  units)  -  4654.9846 


Equation  of  the  line: 

(in  original  units) 

Y  -  to  +  b,X 

4654.9846  -  .449 ,854,82  x 

= _ .005  064 

s»,  =  _  4.458  638 


2, 300 ‘,860 

3,810,502.5 

-1,509,642.5 

(7)  XY2 

2,285,614 

(8)  (Z  Y)*/n 

1,605,289. 

(9)  S„ 

680,325. 

(14)  =  679,119.9614 

&xx  "  * 


(15) 

(n  —  2)  Sy 

=  1,205.0386 

(16) 

sV 

86.074  1857 

Sr 

9.277617 

Estimated  variance  of  the  slope: 

si  =  #-  =  .000  025  649  045 

&XX 

Estimated  variance  of  intercept: 

Si.  -  sV  +  £]}  =  19.879  452 
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Figure  5-8.  Young’s  modulus  of  sapphire  rods  as  a  function 
of  temperature — showing  computed  regression  line 
and  confidence  interval  for  the  line. 


Using  th«  Regression  Equation  for  Prodiction. 

The  fitted  regression  equation  may  be  used  for 
two  kinds  of  predictions: 

(a)  To  estimate  the  true  value  of  y  associated 
with  a  particular  value  of  x,  e.g.,  given  x  -  x'  to 
estimate  the  value  of  y'  =  /30  +  ftx';  or, 

(b)  To  predict  a  single  new  observed  value  Y 
corresponding  to  a  particular  value  of  x,  e.g., 
given  x  =  x'  to  predict  the  value  of  a  single 
measurement  of  y'. 

Which  prediction  should  be  made?  In  some 
cases,  it  is  sufficient  to  say  that  the  true  value  of 
y  (for  given  x)  lies  in  a  certain  interval,  and  in 
other  cases  we  may  need  to  know  how  large  (or 
how  small)  an  individual  observed  Y  value  is 
likely  to  be  associated  with  a  particular  value  of 
x.  The  question  of  what  to  predict  is  similar  to 
the  question  of  what  to  specify  (e.g.,  whether  to 
specify  average  tensile  strength  or  to  specify 
minimum  tensile  strength)  and  can  be  answered 


only  with  respect  to  a  particular  situation.  The 
difference  is  that  here  we  are  concerned  with 
relationships  between  two  variables  and  there¬ 
fore  must  always  talk  about  the  value  of  y,  or  Y, 
for  fixed  x. 

The  predicted  y1  or  Y'  value  is  obtained  by 
substituting  the  chosen  value  (x')  of  x  in  the 
fitted  equation.  For  a  particular  value  of  x, 
either  type  of  prediction  ((a)  or  (b))  gives  the 
same  numerical  answer  for  y'  or  Y‘.  The  un¬ 
certainty  associated  with  the  prediction,  how¬ 
ever,  does  depend  on  whether  we  are  estimating 
the  true  value  of  y‘ ,  or  predicting  the  value  Y' 
of  an  individual  measurement  of  y\  If  the 
experiment  could  be  repeated  many  times,  each 
time  obtaining  »  pairs  of  (x,  Y)  values,  consider 
the  range  of  Y  values  which  would  be  obtained 
for  a  given  x.  Surely  the  individual  Y  values  in 
all  the  sets  will  spread  over  a  larger  range  than 
will  the  collection  consisting  of  the  average  T’s 
(one  from  each  set). 
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To  estimate  the  true  value  of  y  associated 
with  the  value  x',  use  the  equation 

y'c  =  60  +  bix'. 

The  variance  of  y[  as  an  estimate  of  the  true 
value  y'  =  /S0  +  Pi*'  is 

This  variance  is  the  variance  of  estimate  of  a 
point  on  the  fitted  line. 


For  example,  using  the  equation  relating 
Young’s  modulus  to  temperature,  we  predict  a 
value  for  y  at  x  =  1200: 


y'c  =  4654.9846  -  .44985482  (1200) 
y'c  =  4115.16 


Var  y’c  =  86.074 


.0625  + 


(1200  -  751.875)5" 
3,355,843.75 


=  86.074  (.0625  +  .0598) 
-  86.074  (.1223) 

Var  y’c  =  10.53  . 


To  predict  a  single  observed  value  of  Y  corre¬ 
sponding  to  a  given  value  (x')  of  x,  use  the  same 
equation 

Y'c  =  6o  +  blX'. 

The  variance  of  Y’c  as  an  estimate  of  a  single 
new  (additional,  future)  measurement  of  y'  is 

Var  Y!  -  4„  [l  +  I  + 

The  equation  for  our  example  is 
Y  =  4654.9846  -  .44985482  x. 


To  predict  the  value  of  a  single  determination  of 
Young’s  modulus  at  x  =  750,  substitute  in  this 
equation  and  obtain: 

Y'c  »  4654.9846  -  .44985482  (750) 

=  4317.59 


Var  Y' 


4ri+i  +  (X' 


=  86.074 


1  +  .0625  + 


(750  -  751.875)1] 
3,355,843.75 


=  86.074  (1.0625) 
=  91.45  . 


5-4.1. 2  What  are  the  Confidence  Interval  Esti¬ 
mate*  for:  the  Line  a*  a  Whole;  a 
Point  on  the  Line;  a  Future  Value  of  Y 
Corresponding  to  a  Given  Value  of  x? 

Once  we  have  fitted  the  line,  we  want  to  make 
predictions  from  it,  and  we  want  to  know  how 
good  our  predictions  are.  Often,  these  pre¬ 
dictions  will  be  given  in  the  form  of  an  interval 
together  with  a  confidence  coefficient  associated 
with  the  interval — i.e.,  confidence  interval  esti¬ 
mates.  Several  kinds  of  confidence  interval 
estimates  may  be  made: 

(a)  A  confidence  band  for  the  line  as  a  whole. 

(b)  A  confidence  interval  for  a  point  on  the 
line — i.e.,  a  confidence  interval  for  y'  (the  true 
value  of  y  and  the  mean  value  of  Y)  correspond¬ 
ing  to  a  single  value  of  x  =  x'. 

If  the  fitted  line  is,  say,  a  calibration  line 
which  will  be  used  over  and  over  again,  we  will 
want  to  make  the  interval  estimate  described 
in  (a).  In  other  cases,  the  line  as  such  may  not 
be  so  important.  The  line  may  have  been 
fitted  only  to  investigate  or  check  the  structure 
of  the  relationship,  and  the  interest  of  the 
experimenter  may  be  centered  at  one  or  two 
values  of  the  variables. 

Another  kind  of  interval  estimate  sometimes 
is  required: 

(c)  A  single  observed  value  (Y')  of  Y  corre¬ 
sponding  to  a  new  value  of  x  =  x'. 

These  three  kinds  of  confidence  interval  state¬ 
ments  have  somewhat  different  interpretations. 
The  confidence  interval  for  (b)  is  interpreted  as 
follows: 

Suppose  that  we  repeated  our  experiment  a 
large  number  of  times.  Each  time,  we  obtain  n 
pairs  of  values  (x, ,  Yd,  fit  the  line,  and  compute 
aconfidenceintervalestimatefor#'  =  /30  +  0ix\ 
the  value  of  y  corresponding  to  the  particular 
value  x  =  x'.  Such  interval  estimates  of  y'  are 
expected  to  be  correct  (i.e.,  include  the  true 
value  of  y')  a  proportion  (1  —  a)  of  the  time. 
If  we  were  to  make  an  interval  estimate  of  y " 
corresponding  to  another  value  of  x  =  x" ,  these 
interval  estimates  also  would  be  expected  to 
include  y "  the  same  proportion  (1  -  a)  of  the 
time.  However,  taken  together,  these  intervals 
do  not  constitute  a  joint  confidence  statement 
about  y'  and  y"  which  would  be  expected  to 
be  correct  exactly  a  proportion  (1  —  a)  of  the 
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time;  nor  is  the  effective  level  of  confidence 
(1  —  a)1,  because  the  two  statements  are  not 
independent  but  are  correlated  in  a  manner 
Intimately  dependent  on  the  values  x'  and  x"  for 
which  the  predictions  are  to  be  made. 

The  confidence  band  for  the  whole  line  (a) 
implies  the  same  sort  of  repetition  of  the  experi¬ 
ment  except  that  our  confidence  statements  are 
not  now  limited  to  one  a:  at  a  time,  but  we  can 
talk  about  any  number  of  x  values  simultane¬ 
ously — about  the  whole  line.  Our  confidence 
statement  applies  to  the  line  as  a  whole,  and 
therefore  the  confidence  intervals  for  y  corre¬ 
sponding  to  all  the  chosen  x  values  will  simulta¬ 
neously  be  correct  a  proportion  (1  —  a)  of  the 
time.  It  will  be  noted  that  the  intervals  in  (a) 
are  larger  than  the  intervals  in  (b)  by  the  ratio 


■VZF/t.  This  wider  interval  is  the  “price”  we 
pay  for  making  joint  statements  about  y  for  any 
number  of  or  for  all  of  the  x  values,  rather  than 
the  y  for  a  single  x. 

Another  caution  is  in  order.  We  cannot  use 
the  same  computed  line  in  (b)  and  (c)  to  make  a 
large  number  of  predictions,  and  claim  that 
100  (l  —  a)  %  of  the  predictions  will  be  correct. 
The  estimated  line  may  be  very  close  to  the  true 
line,  in  which  case  nearly  all  of  the  interval 
predictions  may  be  correct;  or  the  line  may  be 
considerably  different  from  the  true  line ,  in 
which  case  very  few  may  be  correct.  In  prac¬ 
tice,  provided  our  situation  is  in  control,  we 
should  always  revise  our  estimate  of  the  line  to 
include  additional  information  in  the  way  of 
new  points. 


5-4. 1.2.1  What  it  the  (1  —  a)  Confidence  Band  for  the  Line  at  a  Whole? 


Procedure 

(1)  Choose  the  desired  confidence  level,  1  —  a 


(2)  Obtain  sK  from  Worksheet. 


(3)  Look  up  /V„  for  (2,  n  —  2)  degrees  of  free¬ 
dom  in  Table  A-5. 

(4)  Choose  a  number  of  values  of  X  (within  the 
range  of  the  data)  at  which  to  compute 
points  for  drawing  the  confidence  band. 


(5)  At  each  selected  value  of  X,  compute: 

Yc  =  Y  +  6,  {X  —  X) 

and 

^  *  [s +  1*xrL']‘ 

(6)  A  (1  —  a)  confidence  band  for  the  whole 
line  is  determined  by 

Yc  ±  Wt. 


Example 

(1)  Let:  1  -  a  =  .95 

a  =  .05 

(2)  Sy  =  9.277617 

from  Worksheet  5-4.1 

(3)  F.n  (2,  14)  -  3.74 

(4)  I>>t:  X  =  30 

X  =  400 
X  -  800 
X  =  1200 
X  =  1500, 
for  example. 

(5)  See  Table  5-2  for  a  convenient  computa¬ 
tional  arrangement  and  the  example  cal¬ 
culations. 


(6)  See  Table  5-2. 
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Procedure 

(7)  To  draw  the  line  and  its  confidence  band, 
plot  Y,  at  two  of  the  extreme  selected 
values  of  X.  Connect  the  two  points  by 
a  straight  line.  At  each  selected  value  of 
X,  also  plot  Yc  +  Wi  and  Ye  -  W,.  Con¬ 
nect  the  upper  series  of  points,  and  the 
lower  series  of  points,  by  smooth  curves. 

If  more  points  are  needed  for  drawing  the 
curves  for  the  band,  note  that,  because  of  sym¬ 
metry,  the  calculation  of  Wi  at  n  values  of  X 
actually  gives  W i  at  2 n  values  of  X. 


Example 

(7)  See  Figure  5-8. 


For  example:  W i  (but  not  Y,)  has  the  same 
value  at  X  =  400  (i.e.,  X  —  351.875)  as  at 
X  =  1103.75  (i.e.,  X  +  351.875). 


TABLE  5-2.  COMPUTATIONAL  ARRANGEMENT  FOR  PROCEDURE  5-4.1. 2.1 


X 

(X-X) 

Yc 

1  ,  (X-X)’ 

"  S„ 

H 

*r. 

H 

Yc+  W, 

Y.- 

30 

-721.875 

4641.49 

.21778 

18.7452 

4.3296 

11.84 

4653.33 

4629.65 

400 

-351.875 

4475.04 

.09940 

8.5558 

2.9250 

8.00 

4483.04 

4467.04 

800 

48.125 

4295.10 

.06319 

5.4390 

2.3322 

6.38 

4301.48 

4288.72 

1200 

448.125 

4115.16 

.12234 

10.5303 

3.2450 

8.88 

4124.04 

4106.28 

1500 

748.125 

3980.20 

.22928 

19.7351 

4.4424 

12.15 

3992.35 

3968.05 

X  =  751.875 

=  86.0741857 

Yc  =  Y  +  6,  (X  —  X) 

codhd  Y  =  316.75 

-1-  =  .0625 
n 

4.-4[i  +  <Xs/) 

Y  (original  \inits)  =  4316.75 

6,  -  -  .44985482 

W,  =  2.735  Sr, 

S„  =  3, 35(5,843.75 

V2F  =  2.735 
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T 


5-4.1. 2.2  Give  a  (1  —  a)  Confidence  Interval  Eitimate  for  a  Single  Point  on  the  Line  (i.e.,  the  Mean 
Value  of  y  Corresponding  to  a  Chosen  Value  of  jr  ■*  x') 


Procedure 


Example 


(1)  Choose  the  desired  confidence  level,  1  —  a 


(2)  Obtain  sy  from  Worksheet. 


(3)  Look  up  fi_/3  for  n  —  2  degrees  of  freedom 
in  Table  A-4. 


(4)  Choose  X',  the  value  of  X  at  which  we 
want  to  make  an  interval  estimate  of  the 
mean  value  of  Y. 


(5)  Compute: 

ri  (X'~  xyy 
w3  -  sk  +  -  5-  J 

and 

Yc  =  Y  +  61  (X'  -  X) 


(6)  A  (1  —  «)  confidence  interval  estimate  for 
the  mean  value  of  Y  corresponding  to 
X  -  X'  is  given  by 
Yc  ±  W,. 


(1)  Let:  1  —  a  -  .95 

a  -  .05 

(2)  sr  =  9.277617 

from  Worksheet  5-4.1 

(3)  f..„  (14)  -  2.145 


(4)  Let  X'  =  1200 


(5) 

Wt  -  2.145  (3.2451) 
=  6.96 

Y„  =  4115.16 


(6)  A  95%  confidence  interval  estimate  for  the 
mean  value  of  Y  corresponding  to  X  =  1200 
is 

4115.16  ±  6.96 
-  4108.20  to  4122.12. 


Note:  An  interval  estimate  of  the  intercept  of  the  line  (0O)  is  obtained  by  setting  X'  =  0  in  the 
above  procedure. 
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5-4.1. 2.3  Give  a  (1  —  a)  Confidence  Interval  Estimate  for  a  Single  (Future)  Value  (T)  of  Y  Corre¬ 
sponding  to  a  Chosen  Value  (x')  of  x. 

Procedure  -Example 

(1)  Choose  the  desired  confidence  level,  1  —  a  (1)  Let:  1  —  a  =  .95 

a  =  .05 


(2)  Obtain  sy  from  Worksheet. 

(3)  Look  up  ft-o/j  for  n  —  2  degrees  of  freedom 
in  Table  A-4. 

(4)  Choose  X',  the  value  of  X  at  which  we 
want  to  make  an  interval  estimate  of  a 
single  value  of  Y. 

(5)  Compute: 

W,  =  t^n  sy  [l  +  -  +  J 

and 

Ye  =  Y  +  ft  ( X '  -  X) 

(6)  A  (1  —  a)  confidence  interval  estimate  for 
Y'  (the  single  value  of  Y  corresponding  to 
X')  is 

Y.±W,. 


(2)  Sk  =  9.277617 
from  Worksheet  5-4.1 

(3)  f.,„  (14)  =  2.145 

(4)  Let  X'  =  1200 


(5) 

W ,  -  2.145  (9.8288) 

=  21.08 

Ye  =  4115.16 

(6)  A  95%  confidence  interval  estimate  for 
a  single  value  of  Y  corresponding  to 
X'  =  1200  is 

4115.16  ±  21.08 
=  4094.08  to  4136.24  . 


5-4.1. 3  What  is  the  Confidence  Interval  Estimate  for  ft  ,  the  Slope  of  the  True  Line  y  =  ft  +  ftx? 


(1) 

Procedure 

Choose  the  desired  confidence  level,  1  -  a 

(1) 

Example 

Let:  1  —  a  =  .95 
a  =  .05 

(2) 

Look  up  for  n  —  2  degrees  of  freedom 

in  Table  A-4. 

(2) 

t.m  04)  =  2.145 

(3) 

Obtain  s»,  from ’Worksheet. 

(3) 

s»,  =  .005064 
from  Worksheet  5.4.1 

(4) 

Compute 

(4) 

Sfr, 

W*  «  2.145  (.005064) 
=  .010862 

(5) 

A  (1  —  a)  confidence  interval  estimate  for 
ft  is 

(5) 

ft  =  -  .449855 

Wt  =  .010862 

A  95%  confidence  interval  for  ft  is  the  in¬ 
terval  -.449855  ±  .010862,  i.e.,  the  inter¬ 
val  from  -  .460717  to  -  .438993  . 
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5-4. 1.4  If  W*  Obsarv*  n'  N«w  Values  of  Y  (with  Average  Y'),  How  Can  We  Use  the  Fitted  Regression 
Line  to  Obtain  an  Interval  Estimate  of  the  Value  of  *  that  Produced  These  Values  of  Y? 


Example:  Suppose  that  we  obtain  10  new  measurements  of  Young's  modulus  (with 
average,  Y'  =  4500)  and  we  wish  to  use  the  regression  line  to  make  an  interval  estimate 
of  the  temperature  (x)  at  which  the  measurements  were  made. 

Procoduro  Example 

(1)  Choose  the  desired  confidence  level,  1  —  a  (1)  Let:  1  —  a  =  .95 

a  =  .05 


(2)  Look  up  for  n  —  2  degrees  of  freedom  (2)  /.»7i  (14)  =  2.145 

in  Table  A-4. 


(3)  Obtain  6,  and  s’,  from  Worksheet. 


(3)  From  Worksheet  5-4.1, 
ft,  =  -.449855 

s\,  =  .0000256490 


(4)  Compute 

C  =  ft,  —  (fl-a/j)’  S&l 


(4) 


C  =  .202370  -  .000118 
=  .202252 


(5)  A  (1  —  a)  confidence  interval  estimate  for 
the  X  corresponding  to  Y‘  is  computed 
from 


X' 


X  + 


(5)  A  95%  confidence  interval  would  be  com¬ 
puted  as  follows: 


X'  =  751.875  -  - 


.449855  (4500  -  4316.75) 


.202252 


2.145  (9.277617) 

*  .202252  . .  X 


yliMMk  +  <-1625>  (•2IB252) 

=  751.875  -  407.590 


±  98.39452  V.0100066  +  .0328660 
=  344.285  ±  98.39452  \/ 1)4 28726 
=  344.285  ±  98.39452  (.20706) 

=  344.285  ±  20.374 


The  interval  from  X  -  323.911  to  X  = 
364.659  is  a  95%  confidence  interval  for  the 
value  of  temperature  which  produced  the 
10  measurements  whose  mean  Young’s 
modulus  was  4500. 
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5-4.1. 5  Using  th«  Fitted  Regression  Line,  How  Con  We  Choose  a  Value  (x')  of  x  Which  We  May 
Expect  with  Confidence  (1  —  a)  Will  Produce  a  Value  of  Y  Not  Less  Than  Some  Specified 
Value  Q? 


Example:  What  value  (*;)  of  temperature  (i)  can  be  expected  to  produce  a  value  of 
Young’s  modulus  not  less  than  4300? 


Procedure 


Example 


(1)  Choose  the  desired  confidence  level,  1  —  a;  (1)  Let:  1  —  a  =  .95 

and  choose  Q  a  =  .05 

Q  =  4300 

(2)  Look  up  for  n  —  2  degrees  of  freedom  (2)  (14)  =  1.761 

in  Table 

(3)  Obtain  5,  and  si  from  Worksheet.  (3)  From  Worksheet  5-4.1, 

6,  =  -.449855 

slt  «  .0000256490 


(4)  Compute 

C  =  b{  -  («,_)*  a*, 


(4) 


C  =  .202370  -  .000080 
-  .202290 


(5)  Compute 


(5)  The  value  of  X'  is  computed  as  follows: 


*’-*  +  '*(5Lcrf) 

.  ti-Sy  l(Q~~rj * 


+ 


C't,)c 


where  the  sign  before  the  last  term  is  +  if 
b,  is  positive  or  —  if  &i  is  negative.  We 
have  confidence  (1  —  a)  that  a  value  of 
X  =  X'  will  correspond  to  (produce)  a 
value  of  Y  not  less  than  Q.  (See  discussion 
of  “confidence”  in  straight-line  prediction 
in  Paragraph  5-4. 1.2). 


X'  =  751.875 

-.449855  (4300  -  4316.75) 
+  "  *  .202290 

_  1.761  (9.277617) 

.202290 

|(4300~—  4316.75) 8  ,  /17\~ 
\  3,355,843.75  +\16/(' 


=  751.875  +  37.249 


-  80.764662  V.000084  +  .214933 


=  751.875  +  37.249 
-  80.764662  V.215017 


=  751.875  -f  37.249  -  37.450 
=  751.674 
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5-4.1. 6  Is  fhe  Assumption  of  Unoar  Regression  Justified? 


This  involves  a  test  of  the  assumption  that  the  mean  Y  values  ( Yz)  for  given  x  values  do  lie  on  a 
straight  line  (we  assume  that  for  any  given  value  of  x,  the  corresponding  individual  Y  values  are 
normally  distributed  with  variance  a\ ,  which  is  independent  of  the  value  of  a;).  A  simple  test  is 
available  provided  that  we  have  more  than  one  observation  on  Y  at  one  or  more  values  of  x. 
Assume  that  there  are  n  pairs  of  values  (x,  ,  Y,),  and  that  among  these  pairs  there  occur  only  k 
values  of  x  (where  k  is  less  than  n). 


For  example,  see  the  data  recorded  in  Table  5-3  which  shows  measurements  of  Young’s  modulus 
(coded)  of  sapphire  rods  as  a  function  of  temperature. 


Each  x  is  recorded  in  Column  1,  and  the  corresponding  Y  values  (varying  in  number  from  1  to  3 
in  the  example)  are  recorded  opposite  the  appropriate  x.  The  remaining  columns  in  the  table 
are  convenient  for  the  required  computations. 


TABLE  5-3.  COMPUTATIONAL  ARRANGEMENT  FOR  TEST  OF  LINEARITY 


X 

=  Tem¬ 
per¬ 
ature 

Y 

=  Young’s 
Modulus 
Minus  3000 

ZY 

(ZY)2 

2Y2 

n.X. 

n<X,2 

ZXY 

(ZY)2 

n< 

500 

328 

328 

107584 

107584 

1 

500 

250000 

164000 

107584 

550 

296 

296 

87616 

87616 

1 

550 

302500 

162800 

87616 

600 

266 

266 

70756 

70756 

1 

600 

360000 

159600 

70756 

603 

260 

244 

504 

254016 

127136 

2 

1206 

727218 

303912 

127008 

650 

240 

232 

213 

685 

469225 

156793 

3 

1950 

1267500 

445250 

156408.3 

700 

204 

203 

184 

591 

349281 

116681 

3 

2100 

1470000 

413700 

116427 

750 

174 

175 

154 

503 

253009 

84617 

3 

2250 

1687500 

377250 

84336.3 

800 

152 

146 

124 

422 

178084 

59796 

3 

2400 

1920000 

337600 

59361.3 

850 

117 

94 

211 

44521 

22525 

2 

1700 

1445000 

179350 

22260.5 

900 

97 

61 

158 

24964 

13130 

2 

1800 

1620000 

142200 

12482 

950 

38 

38 

1444 

1444 

1 

950 

902500 

36100 

1444 

1000 

30 

5 

35 

1225 

925 

2 

2000 

2000000 

35000 

612.5 

TOTAL 

4037 

849003 

24 

18006 

13952218 

2756762 

846296 

=  T, 

' 

=  r ,  | 

n 

=  Tt 

-7’. 

-T, 

J 

=  T, 
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Procedure 


Example 


(1)  Choose  a,  the  significance  level  of  the  test. 

(2)  Compute: 

n 

T 

X  =  the  weighted  average  of  X. 

(3)  Compute 


Si  =  Tt  — 


{TiY 

n 


(4)  Compute 


r.  - 


T,  Ti 


n 


T.-IZ& 

n 


(5)  Compute 

fill) 


(6)  Compute 
S ,  =  Tx 


(Tt)' 

n 


(7)  Look  up  f for  (k  —  2,  n  —  k)  degrees  of 
freedom  in  Table  A-5. 


(1)  Let:  *  =  .05 

l  -  «  -  .05 

(2) 

V  _  40 37 
r  ~  24 
=  168.21 

?  _  18006 
A  "  24 
=  750.25 

(3) 

=  679057.04 
n 

Si  =  846296  -  679057.04 
=  167238.96 

(4) 


,  2756762  -  3028759.25 

13952218  -  13509001.5 

-271997.25 

443216.5 

=  -0.6136894 

(5) 

5,  «  -0.6136894  (-271997.25) 
=  166921.83 


(6) 

St  =  849003  -  679057.04 
=  169945.96 

(7)  n  =  24 
k  =  12 

F.„  for  (10,  12)  degrees  of  freedom  =  2.75 


(8)  Compute 


_  /317.13X  /24  -  12\ 
"  V  2707  A  10  / 

-  (.11715)  (1.2) 

-  0.14 


(9)  If  F  >  Fi^, ,  decide  that  the  “array  means”  (9)  Since  F  is  less  than  f  ,  the  hypothesis  of 
Y,  do  not  lie  on  a  straight  line.  Iff  <  F  ,  linearity  is  not  disproved, 

the  hypothesis  of  linearity  is  not  disproved. 
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5-4.2  FI  RELATIONSHIPS  WHEN  THE  INTERCEPT 
IS  KNOWN  TO  BE  EQUAL  TO  ZERO  {LINES 
THROUGH  THE  ORIGIN) 

In  Paragraph  5-4.1,  we  assumed: 

(a)  that  there  is  an  underlying  linear  func¬ 
tional  relationship  between  x  and  y  of  the  form 
y  =  p0  +  piX,  with  intercept  p0  and  slope  ft 
both  different  from  zero ; 

(b)  that  our  data  consist  of  observed  values 

Y i,  Y2,  .  .  . ,  Y„  of  y,  corresponding  to  accu¬ 
rately-known  values  Xi,  xt, _ _  x„  of  x;  and, 

(c)  that  the  Y’s  can  be  regarded  as  being 
independently  and  normally  distributed  with 
means  equal  to  their  respective  true  values  (i.e, 
mean  of  Y,  =  ft  +  ft i  =  1,  2,  . . . ,  n)  and 
constant  variance  a\.x  -  o'1  for  all  x. 

Furthermore,  we  gave:  'a  procedure  (Para¬ 
graph  5-4. 1.2.2  with  X'  =  0)  for  determining 
confidence  limits  for  ft,  and  hence  for  testing 
the  hypothesis  that  p0  -  0,  in  the  absence  of 
prior  knowledge  of  the  value  of  ft;  and  a  proce¬ 
dure  that  is  independent  of  the  value  of  ft 
(Paragraph  5-4.1.3)  for  determining  confidence 
limits  for  Pi ,  and  hence  for  testing  the  hypoth¬ 
esis  that  pi  —  0. 

We  now  consider  the  analysis  of  data  corre¬ 
sponding  to  an  FI  structural  relationship  when 
it  is  known  that  y  =  0  when  x  =  0,  so  that  the 
line  must  pass  through  the  origin,  i.e.,  when  it  is 
known  that  ft  =  0.  To  begin  with,  we  assume 
as  in  (b)  and  (c)  above,  that  our  data  consist  of 
observed  values  Yj ,  Ys , . .  . ,  Y„ ,  of  a  dependent 
variable  y  corresponding  to  accurately-known 
values  x, ,  x, ,  .  .  . ,  x„  of  the  independent  variable 
x  and  that  these  Y’s  can  be  regarded  as  being 
independently  and  normally  distributed  with 
means  fta;, ,  fta:s ,  ,  PiXn ,  respectively,  and 

variances  oy.x  that  may  depend  on  x.  We 
consider  explicitly  the  cases  of  constant  vari¬ 
ance  (<rf.r  —  <r2),  variance  proportional  to 
x  [o\.x  —  ro*),  and  standard  deviation  propor¬ 
tional  to  x  ( <r j' . x  =  To).  Finally,  we  consider 
briefly  the  case  of  cumulative  data  where 
X\  <  x2  <  .  .  .  <  x„  and  the  error  in  Y,  is  of  the 
form  et  +  e2  +  ...  +  e._,  +  e, ,  that  is,  is  the 
sum  of  the  errors  of  all  preceding  Y’s  plus  a 
“private  error”  e,  of  its  own.  Following 
Mande.],(8>  we  assume  that  the  errors  (e.)  are 
independently  and  normally  distributed  with 
zero  means  and  with  variances  proportional  to 
the  length  of  their  generation  intervals,  i.e., 


al  -  (*.  —  x,-- i)(T}.  Under  these  circumstances, 
the  Y’s  will  be  normally  distributed  with  means 
PiXL ,  p&i , ,  piXn ,  respectively,  as  before;  and 
with  variances  <ry{  =  x,o*,  respectively;  but  will 
not  be  independent  owing  to  the  overlap  among 
their  respective  errors. 

5-4.2.1  Line  Through  Origin,  Variance  of  Y ’* 
Independent  of  x.  The  slope  of  the 
best-fitting  line  of  the  form  Y  =  61*  is  given  by 


Yt- 

=  — - 

i-l 

and  the  estimated  variance  of  61  is 

sV 


*  = 
where 


i-l 


S(Y, ■  -  6,  xty 

i-l 


Consequently,  we  may  effect  a  simplification  of 
our  Basic  Worksheet — see  Worksheet  5-4.2.1. 

Using  the  values  of  b ,  and  s4,  so  obtained, 
confidence  limits  for  Pi ,  the  slope  of  the  true  line 
through  the  origin,  y  =  Pix,  can  be  obtained  by 
following  the  procedure  of  Paragraph  5-4.1.3 
using  t i-a/t  for  n  —  1  degrees  of  freedom.  Con¬ 
fidence  limits  for  the  line  as  a  whole  then  are 
obtained  simply  by  plotting  the  lines  y  =  p’jx 
and  y  =  P{-x,  where  P['  and  p2  are  the  upper  and 
lower  confidence  limits  for  Pi  obtained  in  the 
manner  just  described.  The  limiting  lines,  in 
this  instance,  also  furnish  confidence  limits  for 
the  value  y'  of  y  corresponding  to  a  particular 
point  on  the  line,  say  for  x  =  x' ,  so  that  an 
additional  procedure  is  unnecessary.  Confi¬ 
dence  limits  for  a  single  future  observed  Y  corre¬ 
sponding  to  x  =  x'  are  given  by 

bix'  ±  VsV+  (z')1  si,  , 

where  s?-  and  s6,  are  from  our  modified  work¬ 
sheet  and  corresponds  to  n  —  1  degrees  of 
freedom. 
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WORKSHEET  5-4.2.1 

WORKSHEET  FOR  FI  RELATIONSHIPS  WHEN  THE  INTERCEPT  IS  KNOWN  TO  BE  ZERO 
AND  THE  VARIANCES  OF  THE  r*  IS  INDEPENDENT  OF  x 


X  denotes 

XX  = _ 

X  =  _ 


(2)  XX 2 


(3)  2  Y2 


(4)  b 


_  XXY 
1  ~  2X»  ~ 


Y  denotes 

xy  = _ 

f  = _ 


Number  of  points:  n  = 


Step  (1)  2 XY 


(5) 


(2  XYy 

XX* 


(6)  (n  -  1)  4 


Step  (3)  —  Step  (5) 


Step  (1)  -r-  Step  (2)  (7)  sV 


Step  (6)  j-  (n  -  1) 


Equation  of  the  Line: 

Y  =  biX 


Sy  — - 

Estimated  variance  of  the  slope: 

si,  =  =  Step  (7)  -h  Step  (2) 

sh  ~  - - - 


5-4.2.2  Line  Through  Origin,  Variance  Propor¬ 
tional  to  x  (a\.x  —  xtr").  The  slope  of 
the  best-fitting  line  of  form  Y  =  b\X  is  given  by 


n 


the  ratio  of  the  averages,  and  the  estimated 
variance  of  6 ,  is 


±x< 

i-1 


where 


Using  the  values  of  b ,  and  sbl  so  obtained, 
confidence  limits  for  ft ,  the  slope  of  the  true  line 
through  the  origin,  y  =  ftac,  can  be  obtained  by 
following  the  procedure  of  Paragraph  5-4.1.3 
using  L-«/a  for  n  -  1  degrees  of  freedom.  Con¬ 
fidence  limits  for  the  line  as  a  whole  then  are 
obtained  simply  by  plotting  the  lines  y  = 
and  y  =  where  ftv  and  ftL  are  the  upper  and 
lower  confidence  limits  for  ft  obtained  in  the 
manner  just  described.  The  limiting  lines,  in 
this  instance,  also  furnish  confidence  limits  for 
the  value  y'  corresponding ‘to  a  particular  point 
on  the  line,  say  for  x  =  x'.  Confidence  limits 
for  a  single  future  observed  Y  corresponding  to 
x  =  x’,  are  given  by 

bix'  db  <!_/ s  vV  s1  + 

where  sh  is  computed  as  shown  above  and 
corresponds  to  n  —  1  degrees  of  freedom. 
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S-4.2.3  Line  Through  Origin,  Standard  Devia¬ 
tion  Proportional  fox  (<ry.t  =  x<r).  The 
slope  of  the  best-fitting  line  of  form  Y  =  6tx  is 
given  by 

bi  =  ± 

i-i 

the  average  of  the  ralk 
and  the  estimated  variance  of  bi  is 


s?, 


?! 

n 


where 


(n  - 

D«2  =  ±(^Y 

i-i  Va  / 

that  is, 

±R,  - 

e2 

n  (n  - 

Y, 

for 

fl,  =  . 

[Sffl 


n 


(2  Rt)1 
n 


Using  the  values  of  b \  and  sk,  so  obtained, 
confidence  limits  for  0l ,  the  slope  of  the  true  line 
through  the  origin,  y  -  /3iX,  can  be  obtained  by 
following  the  procedure  of  Paragraph  5-4. 1.3 
using  t\^*n  for  n  —  1  degrees  of  freedom.  Con¬ 
fidence  limits  for  the  line  as  a  whole  are  then 
obtained  simply  by  plotting  the  lines  y  —  0['x 
and  y  =  pfx  where  0['  and  are  the  upper  and 
lower  confidence  limits  for  di  obtained  in  the 
manner  just  described.  The  limiting  lines,  in 
this  instance,  also  furnish  confidence  limits  for 
the  value  y'  of  y  corresponding  to  a  particular 
point  on  the  line,  say  for  x  =  x'.  Confidence 
limits  for  a  single  future  observed  Y  correspond¬ 
ing  to  x  ~  x',  are  given  by 

biX'  ±  li-a/a  x'  Vs2  +  sg,  , 

where  s*,  is  computed  as  shown  above  and  f,_„/2 
corresponds  to  n  —  1  degrees  of  freedom. 


S-4.2.4  Line  Through  Origin,  Errors  of  Y ’* 
Cumulative  (Cumulative  Data).  In 

many  engineering  tests  and  laboratory  experi¬ 
ments  the  observed  values  Y, ,  Y2, ... , 
of  a  dependent  variable  y  represent  the  cumula¬ 
tive  magnitude  of  some  effect  at  successive 
values  x,  <  x2  <  z3  <  .  .  .  of  the  independent 


variable  z.  Thus,  Y, ,  Yt, ... ,  may  denote:  the 
total  weight  loss  of  a  tire  under  road  test, 
measured  at  successive  mileages  x, ,  xs,  .  .  .  ;  or 
the  weight  gain  of  some  material  due  to  water 
absorption  at  successive  times  xt ,  xt , . . .  ;  or  the 
total  deflection  of  a  beam  (or  total  compression 
of  a  spring)  under  continually  increasing  load, 
measured  at  loads  x{ ,  x2 , .  . .  ;  and  so  forth.  In 
such  cases,  even  though  the  underlying  func¬ 
tional  relationship  takes  the  form  of  a  line 
through  the  origin,  y  =  (3x,  none  of  the  pro¬ 
cedures  that  we  have  presented  thus  far  will  be 
applicable,  because  of  the  cumulative  effect  of 
errors  of  technique  on  the  successive  Y’s;  the 
deviation  of  Y,  from  its  true  or  expected  value 
y,  will  include  the  deviation  (Y,_i  —  y,-_ t)  of 
Y(_i  from  its  true  or  expected  value,  plus  an 
individual  “private  deviation  or  error”  e,  of  its 
own.  Hence,  the  total  error  of  Y,  will  be  the 
sum  (et  +  e2  +  . . .  +  e,_i  +  e,)  of  the  indi¬ 
vidual  error  contributions  of  Y1(  Y2, .  .  . ,  Y,_s, 
and  its  own  additional  deviation. 

If  the  test  or  experiment  starts  at  x0  =  0, 
and  the  x’s  form  an  uninterrupted  sequence 
0  <  xt  <  x2  <  .  .  .  <  x„ ,  and  if  we  may  regard 
the  individual  error  contributions  et ,  e2 , . .  .  ,  as 
independently  and  normally  distributed  with 
zero  means  and  variances  proportional  to  the 
lengths  of  the  x-intervals  over  which  they  ac¬ 
crue,  i.e.,  if  <rli  =  (x,  —  x,_i)  <72,  then  the  best 
estimate  of  the  slope- of  the  underlying  linear 
functional  relation  y  =  dix  is  given  by 


and  estimated  variance  of  b2 

sl  = _ 1 _ If,  (Y,-  -  _  n 

1  (n  -  1)  x„  x,  -  x,_!  x„ 
in  which  x0  =  0  and  Y0  =  0  by  hypothesis. 

Using  the  values  of  b i  and  sb,  so  obtained, 
confidence  limits  for  0i ,  the  slope  of  the  true  line 
through  the  origin,  y  =  0|X,  can  be  obtained  by 
following  the  procedure  of  Paragraph  5-4. 1.3 
using  ti-a/i  for  n  —  1  degrees  of  freedom.  Con¬ 
fidence  limits  for  the  line  as  a  whole  then  are 
obtained  simply  by  plotting  the  lines  y  =  filfx 
and  y  =  jS/'x,  where  0,"  and  /3 /'  are  the  upper  and 
lower  confidence  limits  for  0,  obtained  in  the 
manner  just  described.  These  limit  lines  also 
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provide  confidence  limits  for  a  particular  point 
on  the  line,  say  the  value  y'  corresponding  to 
x  -  x' .  For  the  fitting  of  lines  of  this  sort  to 
cumulative  data  under  more  general  conditions, 
and  for  other  related  matters,  see  Mandel's 
article.”' 

5-4.3  FH  RELATIONSHIPS 

Distinguishing  Features.  There  is  an  under¬ 
lying  mathematical  (functional)  relationship  be¬ 
tween  the  two  variables,  of  the  form 

y  =  0o  +  0tx. 

Both  X  and  Y  are  subject  to  errors  of  measure¬ 
ment.  Read  Paragraph  5-3.1  and  Table  5-1. 

The  full  treatment  of  this  case  depends  on  the 
assumptions  we  are  willing  to  make  about  error 
distributions.  For  complete  discussion  of  the 
problem,  see  Acton.”' 

5-4.3. 1  A  Simple  Method  of  Fitting  the  Line  In 
the  General  Case.  There  is  a  quick 
and  simple  method  of  fitting  a  line  of  the  form 
Y  =  ba  +  biX  which  is  generally  applicable 
when  both  X  and  Y  are  subject  to  errors  of 
measurement.  This  method  is  described  in 
Bartlett/6'  and  is  illustrated  in  this  paragraph. 
Similar  methods  had  been  used  previously  by 
other  authors. 

(a)  For  the  location  of  the  fitted  straight  line, 
use  as  the  pivot  point  the  center  of  gravity  of 
all  n  observed  points  (Ar,  ,  T,),  that  is,  the  point 
with  the  mean  coordinates  (X,  Y).  In  conse¬ 
quence,  the  fitted  line  will  be  of  the  form 
7  =  60  +  b ,X  with  b0  =  Y  -  b,X,  just  as  in 
the  least-squares  method  in  Paragraph  5-4.1. 

(b)  For  the  slope,  divide  the  w  plotted  points 
into  three  non-overlapping  groups  when  con¬ 
sidered  in  the  X  direction.  There  should  be  an 
equal  number  of  points,  k,  in  each  of  the  two 

71 

extreme  groups,  with  k  as  close  to  ^  as  possible. 
Take,  as  the  slope  of  the  line, 


where 

Yt  =  average  Y  for  3rd  group 
Y i  =  average  Y  for  1st  group 
X 5  =  average  X  for  3rd  group 
JYi  =  average  X  for  1st  group. 


Data  Sample  5-4.3. 1 — Relation  of  Two 
Colorimetric  Method* 

The  following  data  are  coded  results  of  two 
colorimetric  methods  for  the  determination  of  a 
chemical  constituent.  (The  data  have  been 
coded  for  a  special  purpose  which  has  nothing 
to  do  with  this  illustration).  The  interest  here, 
of  course,  is  in  the  relationship  between  results 
given  by  the  two  methods,  and  it  is  presumed 
that  there  is  a  functional  relationship  with  both 
methods  subject  to  errors  of  measurement. 


Sample 

Method  I 
X 

Method  II 

y 

1 

3720 

5363 

2 

4328 

6195 

3 

4655 

6428 

4 

4818 

6662 

5 

5545 

7562 

6 

7278 

9184 

7 

7880 

10070 

8 

10085 

12519 

9 

11707 

13980 

(a)  The  fitted  line  must  pass  through  the 
point  (X,  Y),  where 

X  *  6668.4 
f  =  8662.6 

(b)  To  determine  the  slope,  divide  the  points 
into  3  groups.  Since  there  are  9  points,  exactly 
3  equal  groups  are  obtained. 

Yt  =  12190 
y,  «  5995 
X,  -  9891 
X,  =  4234 


*  Xt  -  X, 

12190  -  5995 
9891  -  4234 
6195 
5657 
=  1.0951 
b0  ~  Y  —  b,X 

=  8662.6  -  (6668.4) 

=  1360.0 
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The  fitted  line 

Y  =  1360.0  +  1.0951  X 
is  shown  in  Figure  5-9. 

Procedures  are  given  in  Bartlett<‘>  for  deter¬ 
mining  100  (1  -  a)  %  confidence  limits  for 
the  true  slope  0t;  and  for  determining  a 
100  (1  —  a)  %  confidence  ellipse  for  p0  and  f}t 


jointly,  from  which  100  (1  —  a)  %  confidence 
limits  for  the  line  as  a  whole  can  be  derived. 
For  strict  validity,  they  require  that  the  meas¬ 
urement  errors  affecting  the  observed  Xx  be 
sufficiently  small  in  comparison  with  the  spacing 
of  their  true  values  a:,-  that  the  allocation  of  the 
observational  points  ( X , ,  Y()  to  the  three  groups 
is  unaffected.  These  procedures  are  formally 


Figure  5-9.  Relationship  between  tu)o  methods  of  determining 
a  chemical  constituent — on  FII  relationship. 
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similar  to  those  appropriate  to  the  least-squares 
method  in  FI  situations,  but  involve  more  com¬ 
plex  calculations.  We  do  not  consider  them 
further  here. 

5-4.3.2  An  Important  Exceptional  Cate.  Until 
comparatively  recently  it  was  not  realized  that 
there  is  a  broad  class  of  controlled  experimental 
situations  in  which  both  X  and  Y  are  subject  to 
errors  of  measurement,  yet  all  of  the  techniques 
appropriate  to  the  FI  case  (x’s  accurately 
known,  measurement  errors  affect  the  Y’s  only) 
are  strictly  applicable  without  change. 

As  an  example,  let  us  consider  the  case  of  an 
analytical  chemist  who,  in  order  to  obtain  an 
accurate  determination  of  the  concentration  of 
a  potassium  sulphate  solution,  decides  to  pro¬ 
ceed  as  follows:  From  a  burette  he  will  draw 
off  5,  10,  IB,  and  20  ml  samples  of  the  solution. 
Volume  of  solution  is  his  independent  variable 
x,  and  his  target  values  are  Xi  =  5,  x2  =  10, 
x3  —  15,  and  aq  =  20,  respectively.  The  vol¬ 
umes  of  solution  that  he  actually  draws  off 
Xt,  X2,  X3,  and  X4  will,  of  course,  differ  from 
the  nominal  or  target  values  as  a  result  of 
errors  of  technique,  and  he  will  not  attempt  to 
measure  their  volumes  accurately.  These  four 
samples  of  the  potassium  sulphate  solution  then 
will  be  treated  with  excess  barium  chloride,  and 
the  precipitated  barium  sulphate  dried  and 
weighed.  Let  Ylf  Yt,  Y3 ,  and  Yt  denote  the 
corresponding  yields  of  barium  sulphate.  These 
yields  actually  will  correspond,  of  course,  to  the 
actual  inputs  X, ,  X2 ,  X3>  and  A'< ,  respectively; 
and  will  differ  from  the  true  yields  associated 
with  these  inputs,  say  y,{X i),  yffX2),  y3(X3), 
and  y\{XY),  respectively,  as  a  result  of  errors  of 
weighing  and  analytical  technique.  The  sul¬ 
phate  concentration  of  the  original  potassium 
sulphate  solution  then  will  be  determined  by 
evaluating  the  slope  bi  of  the  best  fitting 
straight  line  Y  =  60  -f  b tx,  relating  the  observed 
barium  sulphate  yields  (Y, ,  Y2,  Ys,  and  Y<)  to 
the  nominal  or  target  volumes  of  solution 
(x, ,  x-2,  x3,  and  x4) — the  intercept  b0  of  the  line 
making  appropriate  allowance  for  the  possibility 
of  bias  of  the  analytical  procedure  resulting  in  a 
non-zero  blank. 

Without  going  into  the  merits  of  the  foregoing 
as  an  analytical  procedure,  let  us  note  a  number 
of  features  that  are  common  to  controlled  experi¬ 


ments:  First,  the  experimental  program  involves 
a  number  of  preassigned  nominal  or  target 
values  (*i,  x2,  .  .  .)  of  the  independent  variable 
x,  to  which  the  experimenter  equates  the  inde¬ 
pendent  variable  in  his  experiment  as  best  he 
can,  and  then  observes  the  corresponding  yields 
(Yi,  Y2,  . . .)  of  the  dependent  variable  y; 
Second,  the  experimenter,  in  his  notebook, 
records  the  observed  yields  ( Yi ,  Yt, .  .  .)  as  corre¬ 
sponding  to,  and  treats  them  as  if  they  were 
produced  by,  the  nominal  or  target  values 
(Xi,  x2, . . .)  of  the  independent  variable — where¬ 
as,  strictly  they  correspond  to,  and  were  pro¬ 
duced  by,  the  actual  input  values  (Xi ,  X2 ,  .  .  .), 
which  ordinarily  will  differ  somewhat  from  the 
nominal  or  target  values  (xi,  x2,  . .  .)  as  a 
result  of  errors  of  technique.  Furthermore,  the 
effective  values  (X, ,  X2, .  . .)  of  the  independent 
variable  actually  realized  in  the  experiment  are 
not  recorded  at  all — nor  even  measured! 

It  is  surprising  but  nevertheless  true  that  an 
underlying  linear  structural  relationship  of  the 
form  y  =  fo  +  fox  can  be  estimated  Validly 
from  the  results  of  such  experiments,  by  fitting 
a  line  of  the  form  Y  =  b0  +  6,x  in  accordance 
with  the  procedures  for  FI  situations  (x’s  known 
accurately,  Y’s  only  subject  to  error).  This 
fact  was  emphatically  brought  to  the  attention 
of  the  scientific  world  by  Joseph  Berkson  in  a 
paper"0  published  in  1950,  and  for  its  validity 
requires  only  the  usual  assumptions  regarding 
the  randomness  and  independence  of  the  errors 
of  measurement  and  technique  affecting  both 
of  the  variables  (i.e.,  causing  the  deviations  of 
the  actual  inputs  Xt ,  X2 ,  .  .  .  ,  from  their  target 
values  x, ,  x2 ,  .  .  . ,  and  the  deviations  of  the  ob¬ 
served  outputs  Y, ,  Y2,  ...,  from  their  true 
values  of  ^(Xi),  y2(X2),  . .  ,).  The  conclusion 
also  extends  to  the  many-variable  case  con¬ 
sidered  in  Chapter  6,  provided  that  the  relation¬ 
ship  is  linear,  i.e.,  that 

y  =  fo  +  fox  -f-  fou  +  0tv  +  .  .  .  . 

If  the  underlying  relationship  is  a  polynomial 
in  x  (e.g.,  y  =  fo  +  PiX  +  fox2  +  fox*),  then 
Geary'7’  has  found  that  Berkson's  conclusion 
carries  over  to  the  extent  that  the  usual  least- 
squares  estimates  (given  in  Chapter  6)  of  the 
coefficients  of  the  two  highest  powers  of  x  (i.e., 
of  fo  and  fo  here)  retain  their  optimum  proper¬ 
ties  of  unbiasedness  and  minimum  variance,  but 
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the  confidence-interval  and  tests-of-significance 
procedures  require  modification. 

B<4.4  SOME  LINEARIZING 
TRANSFORMATIONS 

If  the  form  of  a  non-linear  relationship  be¬ 
tween  two  variables  is  known,  it  is  sometimes 
possible  to  make  a  transformation  of  one  or 
both  variables  such  that  the  relationship  be¬ 
tween  the  transformed  variables  can  be  ex¬ 
pressed  as  a  straight  line.  For  example,  we 
might  know  that  the  relationship  is  of  the  form 
Y  =  abx.  If  we  take  logs  of  both  sides  of  this 
equation,  we  obtain 

log  Y  =  log  a  +  X  log  b, 

which  will  be  recognized  to  be  a  straight  line 
whose  intercept  on  the  log  Y  scale  is  equal  to 
log  a,  and  whose  slope  is  equal  to  log  b.  The 
procedure  for  fitting  the  relationship  is  given  in 
the  following  steps. 

(1)  Make  the  transformation  Yr  =  log  Y 
(i.e.,  take  logs  of  all  the  observed  Y 
values). 

(2)  Use  the  procedure  of  Paragraph  5-4. 1.1 
to  fit  the  line  YT  -  b0  -f  btX,  substi¬ 
tuting  Yr  everywhere  for  Y. 

(3)  Obtain  the  constants  of  the  original 
equation  by  substituting  the  calculated 
values  of  ba  and  bi  in  the  following 
equations: 

60  =  log  a 
b  i  =  log  b, 

and  taking  the  required  antilogs. 

Some  relationships  between  X  and  Y  which 
can  easily  be  transformed  into  straight-line 
form  are  shown  in  Table  5-4.  This  table  gives 
the  appropriate  change  of  variable  for  each  rela¬ 
tionship,  and  gives  the  formulas  to  convert  the 
constants  of  the  resulting  straight  line  to  the 
constants  of  the  relationship  in  its  original  form. 
In  addition  to  the  ones  given  in  Table  5-4,  some 
more-complicated  relationships  can  be  handled 
by  using  special  tricks  which  are  not  described 
here,  but  can  be  found  in  Lipka,(8)  Rietz,(,)  and 
Scarborough. ao) 


It  should  be  noted  that  the  use  of  these  trans¬ 
formations  is  certain  to  accomplish  one  thing 
only — i.e.,  to  yield  a  relationship  in  straight-line 
form.  The  transformed  data  will  not  neces¬ 
sarily  satisfy  certain  assumptions  which  are 
theoretically  necessary  in  order  to  apply  the 
procedures  of  Paragraph  5-4.1. 1,  for  example, 
the  assumption  that  the  variability  of  Y  given 
X  is  the  same  for  all  X.  However,  for  practical 
purposes  and  within  the  range  of  the  data  con¬ 
sidered,  the  transformations  often  do  help  in 
this  regard. 

Thus  far,  our  discussion  has  centered  on  the 
use  of  transformations  to  convert  a  known  rela¬ 
tionship  to  linear  form.  The  existence  of  such 
linearizing  transformations  also  makes  it  pos¬ 
sible  to  determine  the  form  of  a  relationship  em¬ 
pirically.  The  following  possibilities,  adapted 
from  Scarborough, <10)  are  suggested  in  this 
regard: 

(1)  Plot  Y  against  on  ordinary  graph 

paper.  If  the  points  lie  on  a  straight  line,  the 
relationship  is 


(2)  Plot  y  against  X  on  ordinary  graph 

paper.  If  the  points  lie  on  a  straight  line,  the 
relationship  is 

Y  =  a  +  bX  . 

(3)  Plot  X  against  Y  on  semilog  paper  (X  on 
the  arithmetic  scale,  Y  on  the  logarithmic  scale). 
If  the  points  lie  on  a  straight  line,  the  variables 
are  related  in  the  form 

Y  =  aeiX,  or 
7  =  abx  . 

(4)  Plot  Y  against  X  on  log-log  paper.  If 
the  points  lie  on  a  straight  line,  the  variables  are 
related  in  the  form 

7  =  oX*  . 
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TABLE  5-4.  SOME  LINEARIZING  TRANSFORMATIONS 


If  the  Relationship 

It  of  the  Form: 

Plot  the  Trantformed 
Variable* 

i 

1 

Fit  the  Straight  Line 

Yr  -  b»  1  6,X, 

Convert  Straight  Line 
Coiutant*  nnd  h,l 

To  Original  Can»Nml»t 

Yt  = 

Xr  = 

bo  = 

b,  = 

+ 

e 

II 

Y 

1 

X 

Use  the  procedures  of 
Paragraph  5-4. 1.1. 

In  all  formulas  given 
there,  substitute  values 
of  Yr  for  y  and  values 
of  X r  for  X,  as  appro¬ 
priate. 

a  j 

i 

b 

Y  -  ,  or 

a  +  oX 

Y  =  «  +  bX 

1 

V 

X 

1 

1 

: 

a 

b 

Y-  X 

1  a +  bX 

X 

Y 

X 

a 

b 

Y  =  abx 

log  Y 

X 

log  a 

log  b 

Y  =  aebx 

log  Y 

X 

log  a 

b  log  e 

Y  -  aX” 

log  y 

log  X 

log  a 

b 

Y  -  a  +  bX ", 

where  n  is  known 

y 

X" 

a 

b 

5-5  PROBLEMS  AND  PROCEDURES  FOR  STATISTICAL  RELATIONSHIPS 


5-5.1  SI  RELATIONSHIPS 

In  this  case,  we  are  interested  in  an  associa¬ 
tion  between  two  variables.  See  Paragraph 
5-3.2  and  Table  5-1. 

We  usually  make  the  assumption  that  for 
any  fixed  value  of  X,  the  corresponding  values 
of  Y  form  a  normal  distribution  with  means 
Yx  =  /9o  +  0iX  and  variance  <j\.x  (read  as 
“variance  of  Y  given  X”)  which  is  constant  for 
all  values  of  X.  *  Similarly,  we  usually  assume 
that  for  any  fixed  value  of  Y,  the  corresponding 
values  of  X  form  a  normal  distribution  with 
mean  XY  =  0'o  +  0[Y  and  variance  cr\.y  ,  (vari- 

*  Strictly,  we  should  write 

tnY.x  —  0o  +  ft  X 
and 

m.x.Y  =  4-  01  Y  . 

See  Footnote  in  Paragraph  5-3.2. 


ance  of  X  given  Y)  which  is  constant  for  all 
values  of  Y.*  Taken  together,  these  two  sets 
of  assumptions  imply  that  X  and  Y  are  jointly 
distributed  according  to  the  bivariate  normal 
distribution.  In  practical  situations,  we  usually 
have  only  a  sample  from  all  the  possible  pairs 
of  values  X  and  Y,  and  therefore  we  cannot 
determine  either  of  the  true  regression  lines, 
Yx  —  /So  +  PiX  or  Xy  -  fa  +  0[Y,  exactly.  If 
we  have  a  random  sample  of  n  pairs  of  values 
(Xi,  Y i),  ( Xi ,  Tj).  •  ■  •  »  {X„,  y„),  we  can  esti¬ 
mate  either  line,  or  both.  Our  method  of  fitting 
the  line  gives  us  best  predictions  in  the  sense 
that,  for  a  given  X  -  X’  our  estimate  of  the 
corresponding  value  of  Y  -  Y'  will: 

(a)  on  the  average  equal  Yx -  the  mean  value 
of  Y  for  X  =  X'  (i.e.,  it  will  be  on  the  true  line 
Yx  —  (So  -f-  0iX ) ;  and 

(b)  have  a  smaller  variance  than  had  we  used 
any  other  method  for  fitting  the  lind. 
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Data  Sample  5-5.1 — Estimated  Tread  Wear  of  Tires 

The  data  used  for  illustration  are  from  a 
study  of  two  methods  of  estimating  tread  wear 
of  commercial  tires  (Stiehler  and  others(n)). 
The  data  are  shown  here  and  plotted  in  Figure 
5-10.  The  variable  which  is  taken  as  the  inde¬ 
pendent  variable  X  is  the  estimated  tread 
life  in  hundreds  of  miles  by  the  weight-loss 
method.  The  associated  variable  Y  is  the  esti¬ 
mated  tread  life  by  the  groove-depth  method 
(center  grooves).  The  plot  seems  to  indicate  a 
relationship  between  X  and  Y,  but  the  relation¬ 
ship  is  statistical  rather  than  functional  or 
exact.  The  scatter  of  the  points  stems  pri¬ 
marily  from  product  variability  and  variation 
of  tread  wear  under  normal  operating  condi¬ 
tions,  rather  than  from  errors  of  measurement 
of  weight  loss  or  groove  depth.  Descriptions 
and  predictions  are  applicable  only  “on  the 
average.” 


X 

=  Tread  Life 
(Hundreds  of  Miles) 
Estimated  By 
Weight  Method 

y 

=  Tread  Life 
(Hundreds  of  Miles) 
Estimated  By 
Center  Groove  Method 

459 

357 

419 

392 

375 

311 

334 

281 

310 

240 

305 

287 

309 

259 

319 

233 

304 

231 

273 

237 

204 

209 

245 

161 

209 

199 

189 

152 

137 

115 

114 

112 

5-5.1. 1  What  is  tha  Bast  Lin*  To  B«  Used  for  Estimating  Yx  for  Givon  Valuos  of  X? 


Procedure 

The  procedure  is  identical  to  that  of  Paragraph  5-4.1.1.  Using  Basic  Worksheet  (see 
Worksheet  5-5.1),  compute  the  line 

y  -  bo  +  bxX. 


This  is  an  estimate  of  the  true  regression  line 

Yx  =  Po  +  PiX. 

Using  Data  Sample  5-5.1,  the  equation  of  the  fitted  line  is 

7  =  13.506  +  0.790212  X. 

In  Figure  5-11,  the  line  is  drawn,  and  confidence  limits  for  the  line  (see  Paragraph  5-5.1.2) 
are  shown. 
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WORKSHEET  5-3.1 
EXAMPLE  OF  SI  RELATIONSHIP 


X  denotes  Tread  Life  Estimated 
by  Weight  Method 


7  denotes  Tread  Life  Estimated 

by  Center  Groove  Method 


2X  =  4506  _ 27  =  3776 

X  =  281.5625  7  =  236 


Number  of  points:  n  = _ 16 


Step  (1)  2X7 

1,170,731 

(2)  (2X)  (27)/n  = 

1,063,180 

(3)  Szv 

— 

107551 

— 

(4) 

2X’ 

-  1,404,543 

(7) 

27s 

=  986740 

(5) 

(2Xy-/n 

1,268,439.0625 

(8) 

(27)V» 

=  891136 

(6) 

s„ 

-  136103.9375 

(9) 

Syy 

94604 

(10) 

by  = 

S  XV 

Sxx 

.790212 

(14) 

«=  84988.119 

0*x 

(11) 

7 

236 

(15)  (n  -  2)  sV 

=  9616.881 

(12) 

buX 

222.494 

(16)  4. 

686.849 

(13) 

b0  = 

7  -  6iX  = 

13.506 

8y 

-  26.21 

Estimated  variance  of  the  slope: 

«i,  =  ^  =  .005046504 

Estimated  variance  of  intercept: 

<  -  +  £}  - - 


Equation  of  the  line: 

Y  =  bB  +  6,X 

13.506  +  .790212  X 

sbl  = _ _  0.0710387 

St^  = _ _  _  21 . 048_ 
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Figure  5-11.  Relationship  between  weight  method  and  center 
groove  method — the  line  shown  with  its  confidence 
band  is  'for  estimating  tread  life  by  center  groove 
method  from  tread  life  by  weight  method. 


Using  the  Regression  Line  for  Prediction.  The  equation  of  the  fitted  line  may  be  used  to  predict 
¥ x,  the  average  value  of  Y  associated  with  a  value  of  X.  For  example,  using  the  fitted  line, 
Y  =  13.506  +  0.790212  X,  the  following  are  some  predicted  values  for  ¥x. 


X 

¥x 

200 

172 

250 

211 

300 

251 

350 

290 

400 

450 

369 
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5-5. 1.2  What  art  the  Confidence  Interval  Estimate*  for:  the  Line  at  a  Whole;  a  Point  on  the  Line; 
a  Single  Y  Corresponding  to  a  New  Value  of  X? 

Read  the  discussion  of  the  interpretation  of  three  types  of  confidence  intervals  in  Paragraph 
5-4. 1.2,  in  order  to  decide  which  is  the  appropriate  kind  of  confidence  interval. 

The  solutions  are  identical  to  those  given  in  Paragraph  5-4. 1.2,  and  are  illustrated  for  the  tread 
wear  of  commercial  tires  example  (Data  Sample  5-5.1). 


5-5. 1.2.1  What  Is  the  (1  —  a)  Confidence  Band  for  the  Line  as  a  Whole? 


Procedure 

(1)  Choose  the  desired  confidence  level,  1  -  or 

(2)  Obtain  sY  from  Worksheet. 

(3)  Look  up  F  i_„  for  (2,  n  —  2)  degrees  of  free¬ 
dom  in  Table  A-5. 

(4)  Choose  a  number  of  values  of  X  (within  the 
range  of  the  data)  at  which  to  compute 
points  for  drawing  the  confidence  band. 


(5)  At  each  selected  value  of  X,  compute: 

Yc  =  Y  +  by  (X  -  X) 
and 

(6)  A  (1  -  a)  confidence  band  for  the  whole 
line  is  determined  by 

Yt  ±  Wy  . 

(7)  To  draw  the  line  and  its  confidence  band, 
plot  Yc  at  two  of  the  extreme  selected 
values  of  X.  Connect  the  two  points  by  a 
straight  line.  At  each  selected  value  of  X, 
plot  also  Yc  -f  Wy  and  Yc  —  Wy.  Connect 
the  upper  series  of  points,  and  the  lower 
series  of  points,  by  smooth  curves. 

If  more  points  are  needed  for  drawing  the 
curves,  note  that,  because  of  symmetry,  the  cal¬ 
culation  of  W\  at  n  values  of  X  actually  gives 
W  v  at  2 n  values  of  X. 


Example 

(1)  Let:  1  -  a  =  .95 

a  =  .05 

(2)  Sy  -  26.21 

(3)  n  =  16 

F.n  (2,  14)  =  3.74  . 

(4)  Let:  X  «  200 

X  =  250 ' 

X  =  300 
X  «  350 
X  =  400, 
for  example. 

(5)  See  Table  5-5  for  a  convenient  computa¬ 
tional  arrangement,  and  the  example  cal¬ 
culations.  , 


(6)  See  Table  5-5. 


(7)  See  Figure  5-11. 


For  example:  Wy  (but  not  Yc)  has  the  same 
value  at  X  =  250  (i.e.  X  —  31.56)  as  at 
X  =  313.12  (i.e.,  X  -I-  31.56). 
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TABLE  5-5.  COMPUTATIONAL  ARRANGEMENT  FOR  PROCEDURE  5-5.1 .2.1 


X 

(X-X) 

Yt 

1  ,  (X  -  X)* 
n  S„ 

*v. 

w, 

Y.+  W, 

Yt-  W, 

200 

-81.56 

171.6 

0.111375 

76.50 

8.746 

23.9 

195.5 

147.7 

250 

-31.56 

211.1 

0.069818 

47.95 

6.925 

18.9 

230.0 

192.2 

300 

4-18.44 

250.6 

0.064998 

44.64 

6.681 

18.3 

268.9 

232.3 

350 

68.44 

290.1 

0.096915 

66.57 

8.159 

22.3 

312.4 

267.8 

400 

118.44 

329.6 

0.165569 

113.72 

10.66 

t 

29.2 

i 

358.8 

300.4 

X  =  281.5625 
Y  =  236 


s\  =  686.849 

-  =  .06^5 
n 

bi  -  0.790212 
SIX  =  136103.9375 


Y,  =  ?  +  bi  {X  -  X) 

V2F  =  V^48 
=  2.735 


W,  -  V2F  sr. 


5-5.1. 2.2  Give  a  (1  —  a)  Confident*  Interval  Estimate  For  a  Single  Point  On  the  Line,  I.a.,  the  Mean 


Value  of  Y  Corresponding  to  X  <=»  X'. 
Procedure 

(1)  Choose  the  desired  confidence  level,  1  —  a 

(2)  Obtain  Sr  from  Worksheet. 

(3)  Look  up  t^n  for  n  —  2  degrees  of  freedom 
in  Table  A-4. 

(4)  Choose  X',  the  value  of  X  at  which  we 
want  to  make  an  interval  estimate  of  the 
mean  value  of  Y. 

(5)  Compute: 

Tir  —  +  ~  I” T  .  (X1  — 

and 

Yc  =  ?  +  bi  ( X '  -  X) 

(6)  A  (1  —  a)  confidence  interval  estimate  for 
the  mean  value  of  Y  corresponding  to 
X  =  X'  is  given  by 

Y,  ±  Wt  . 


Example 

(1)  Let:  1  —  a  =  .95 

a  =  .05 

(2)  sK  =  26.21 

(3)  n  =  16 

for  14  d.f.  =  2.145 

(4)  Let  X'  -  250, 

for  example. 

(5) 

W2  =  (2.145)  (26.21)  (.2642) 

=  14.85 

Yc  =  211.1 

(6)  A  95%  confidence  interval  estimate  for  the 
mean  value  of  Y  corresponding  to  X  =*  250 
is 

211.1  ±  14.8  , 

the  interval  from  196.3  to  225.9  . 
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5-5. 1.2.3  Give  a  (1  —  a)  Confidence  Interval  Estimate  For  a  Single  (Future)  Value  of  Y  Corresponding 
to  a  Chosen  Value  of  X  =  X'. 

Procedure  Example 

(1)  Choose  the  desired  confidence  level,  1  - 


(2)  Obtain  sY  from  Worksheet. 

(3)  Look  up  for  n  —  2  degrees  of  freedom 
in  Table  A-4. 

(4)  Choose  X',  the  value  of  X  at  which  we 
want  to  make  an  interval  estimate  of  a 
single  value  of  Y. 

(5)  Compute: 

jrr  .  _L  1  ,  (XL  -  *)2~1* 

ws  =  L-./2  [1  +  -  +  -  ~s;x  J 

and 

Yc  =  V  +  bL  (X'  -  X) 

(6)  A  (1  -  a)  confidence  interval  estimate  for 
Y'  (the  single  value  of  Y  corresponding  to 
X')  is 

Yc  ±  W3  . 


(1)  Let:  1  -  «  =  .95 

a  =  .05 

(2)  =  26.21 

(3)  n  =  16 

t.m  for  14  d.f.  =  2.145 

(4)  Let  X'  =  250, 

for  example. 

(5) 

W3  =  (2.145)  (26.21)  (1.0343) 

=  58.1 

Yc  =  211.1  . 

(6 1  A  95%  confidence  interval  estimate  for  a 
single  value  of  Y  corresponding  to  X'  -  250 
is  211.1  ±  58.1,  the  interval  from  153.0  to 
269.2  . 


5-5. 1.3  Give  a  Confidence  Interval  Estimate  For  ,  the  Slope  of  the  True  Regression  Line, 

Yx  -  ft  +  /3.X. 


The  solution  is  identical  to  that  of  Paragraph  5-4. 1.3  and  is  illustrated  here  for  Data  Sample  5-5.1 . 


(1) 

Procedure 

Choose  the  desired  confidence  level,  1  -  a 

(1) 

Example 

Let:  1  —  a  =  .95 

a  =  .05 

(2) 

Look  up  2  for  n  —  2  degrees  of  freedom 

in  Table  A-4. 

(2) 

n  =  16 

t .975  for  14  d.f.  =  2.145 

(3) 

Obtain  s*,  from  Worksheet. 

(3) 

s»,  =  0.0710387 

(4) 

Compute 

(4) 

W  4  =  1 1— a/2  Sb, 

W,  =  (2.145)  (.0710387) 

=  0.152378 

(5) 

A  (1  —  a)  confidence  interval  estimate  for 

J3i  is 

(5) 

6,  =  0.790212 

Wt  =  0.152378 

6.  ±  W<  . 

A  95%  confidence  interval  estimate  for  0, 
is  the  interval  0.790212  ±  0.152378,  i.e., 
the  interval  from  0.637834  to  0.942590  . 
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5-5. 1.4  What  I*  the  Bett  Lin*  For  Predicting  XY 
From  Oiv*n  Values  of  Y7 

For  this  problem,  we  fit  a  line  X  =  bl  +  b[  Y 
(an  estimate  of  the  true  line  Xr  =>  4-  (S[  F). 

To  fit  this  line  we  need  to  interchange  the  roles 
of  the  X  and  F  variables  in  the  computations 
outlined  in  Worksheet  5-5.1  and  proceed  as  in 
Paragraph  5-5.1. 1. 

That  is,  the  fitted  line  will  be: 

X  =  b'c  +  b[Y  , 
where 

=  X  —  6;  F 

and 


From  Data  Sample  5-5.1: 
107551 
1  “  94604 
=  1.136855 


6;  -  281.5625  -  (1.136855)  (236) 

=  13.26 

The  equation  of  the  fitted  line  is: 

X  =  13.26  +  1.136855  Y  , 

and  this  line  is  shown  in  Figure  5-12,  along  with 
the  line  for  predicting  Y  from  X. 

In  order  to  obtain  confidence  intervals,  we 
need  the  following  formulas: 


Figure  5-12.  Relationship  between  weight  method  and  center 
groove  method — showing  the  two  regression  lines. 
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5-5.1. 5  What  it  the  Degree  of  Relationship  of  the  Two  Variables  X  and  Y  as  Measured  by  p ,  the 


Correlation  Coefficient? 
Procedure 

(1)  Compute 

_ Sly _ 

T  ~  VSXZ  VSyy 


(2)  A  95%  confidence  interval  for  p  can  be  ob¬ 
tained  from  Table  A-17,  using  the  appro¬ 
priate  n  and  r.  If  the  confidence  interval 
does  not  include  p  =  0,  we  may  state  that 
the  data  give  reason  to  believe  that  there  is 
a  relationship  (measured  by  p  ^  0)  be¬ 
tween  the  two  variables;  otherwise,  we  may 
state  that  the  data  are  consistent  with  the 
possibility  that  the  two  variables  are  un¬ 
correlated  (p  =  0). 


5-5.2  Sll  RELATIONSHIPS 

In  this  case,  we  are  interested  in  an  associa¬ 
tion  between  two  variables.  This  case  differs 
from  SI  in  that  one  variable  has  been  measured 
at  only  preselected  values  of  the  other  variable. 
(See  Paragraph  5-3.2  and  Table  5-1.) 

For  any  given  value  of  X,  the  corresponding 
values  of  Y  have  a  normal  distribution  with 
mean  Yx  —  0<>  +  @,X,  and  variance  Y  which 
is  independent  of  the  value  of  X.  We  have  n 
pairs  of  values  (Ai ,  7,),  (X, ,  7*), . . . ,  (Xn ,  7„), 
in  which  X  is  the  independent  variable.  (The 
X  values  are  selected,  and  the  7  values  are 
thereby  determined.)  We  wish  to  describe  the 
line  which  will  enable  us  to  make  the  best  esti¬ 
mate  of  values  of  7  corresponding  to  given 
values  of  X. 

We  have  seen  that  for  SI  there  are  two  lines, 
one  for  predicting  7  from  X  and  one  for  pre¬ 
dicting  X  from  7.  When  we  use  only  selected 
values  of  A',  however,  the  only  appropriate  line 
to  fit  is  7  =  ft,,  +  6,  X. 

It  should  be  noted  that  SII  is  handled  com¬ 
putationally  in  the  same  manner  as  FI,  but  both 
the  underlying  assumptions  and  the  interpreta¬ 
tion  of  the  end  results  are  different. 


Example 

(1)  Using  Worksheet  5-5.1, 

_ 107551 

T  V136103.94  V94604 
107551 

(368.92)  (307.58) 

-  0.95  . 

(2)  n  =  16 

r  =  0.95 

From  Table  A-17,  the  95%  confidence  in¬ 
terval  estimate  of  p  is  the  interval  from 
0.85  to  0.98 .  Since  this  interval  does  not 
include  p  =  0,  we  may  state  that  the  data 
give  reason  to  believe  that  there  is  a  rela¬ 
tionship  between  the  two  methods  of  esti¬ 
mating  tread  wear  of  tires. 


Data  Sample  5-5.2 — Estimated  Tread  Wear  of  Tires 

For  our  example,  we  use  part  of  the  data  used 
in  Data  Sample  5-5.1  (the  SI  example).  Sup¬ 
pose  that,  due  to  some  limitation,  we  were  only 
able  to  measure  X  values  between  X  ~  200  and 
X  -  400,  or  that  we  had  taken  but  had  lost  the 
data  for  X  <  200  and  X  >  400.  From  Figure 
5-10,  we  use  only  the  11  observations  whose  X 
values  are  between  these  limits.  The  "se¬ 
lected”  data  are  recorded  in  the  following  table. 


X 

=  Tread  Life 
(Hundreds  of  Miles) 
Estimated  By 
Weight  Method 

7 

=  Tread  Life 
(Hundreds  of  Miles) 
Estimated  By 
Center  Groove  Method 

375 

311 

334 

281 

310 

240 

305 

287 

309 

259 

319 

233 

304 

231 

273 

237 

204 

209 

245 

161 

209 

199 
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5-5.2. 1  What  It  the  Best  Line  To  Be  Used  for 
Estimating  Y.v  From  Given  Values 
of  X? 

Procedure 

Using  Basic  Worksheet  (see  Worksheet 
5-5.2),  compute  the  line  Y  =  6c  -f-  b^X.  This 
is  an  estimate  of  the  true  line  Yx  =  /3o  +  PiX. 


Using  Data  Sample  5-5.2,  the  fitted  line  is 

Y  =  48.9C5  4  0.66L873  X. 

The  fitted  line  is  shown  in  Figure  5-13,  and  the 
confidence  band  for  the  line  (see  the  procedure 
of  Paragraph  5-5.2. 2.1)  also  is  shown. 


WORKSHEET  5-5.2 


EXAMPLE  OF 

Sll  RELATIONSHIP 

X  denotes  Tread  Life 
by  Weight 

Estimated 

Method 

y  denotes  Tread  Life  Estimated  by 

Center  Groove  Method 

ZX 

=  3187 

ZY 

=  2648 

X 

=  289.727 

Y 

=  240.727 

Number  of  points:  n 

=  11 

Step  (1)  ZXY 

_ 

785369 

(2)  (ZX)  ( ZY)/n  = 

767197.818 

(3) 

18171.182 

(4) 

ZX1 

950815 

(7) 

zY2 

655754 

(5) 

{ZX)2/n 

923360.818 

(8) 

( ZY)2/n 

637445.818 

(6) 

s„ 

27454.182 

(9) 

Syy 

18308.182 

(10) 

b  —  SjLV  _ 

bl  -  s» 

0.661873 

(14) 

12027.015 

&xx 

(U) 

Y 

240.727 

(15) 

(n  —  2)  s\  - 

6281.167 

(12) 

bxX 

191.762 

(16) 

697.9074 

(13) 

ba  —  Y  —  biX  - 

48.965 

$Y  ~ 

26.418 

Equation  of  the  line: 

Estimated  variance  of  the  slope: 

y 

-  b„  +  b,X 

4  = 

S\r 

.0254208 

=  48.965  +  0.661873  X 

s„, 

=  0 

159439 

Estimated  variance  of  intercept: 

S&o 

-a 

46.88 

= 

2197.313 
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Figure  5-1 3.  Relationship  between  weight  method  and  center 
groove  method  when  the  range  of  the  weight  method 
has  been  restricted — an  SII  relationship. 


S-5.2.2  What  are  the  Confidence  Interval  Estimates  for:  the  Line  at  a  Whole;  a  Point  on  the  Line; 
a  Single  Y  Corresponding  to  a  New  Value  of  X? 


Read  the  discussion  of  the  interpretation  of  these  three  types  of  confidence  intervals  in  Paragraph 
5-4. 1.2  in  order  to  decide  which  is  the  appropriate  kind  of  confidence  interval. 
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5-5.2. 2.1  What  Is  the  (1  —  a)  Confidence  Band  For  the  Line  as  a  Whole? 

The  solution  is  identical  to  that  of  Procedure  5-4. 1.2.1  and  is  illustrated  here  for  Data  Sample 
5-5.2. 

Procedure  Example 

(1)  Choose  the  desired  confidence  level,  1  —  a  (1)  Let:  1  —  a  —  .95 

«  =  .05 


* 

(2)  Obtain  sr  from  Worksheet.  (2)  From  Worksheet  5-5.2 

sy  =  26.418 


(3)  Look  up  Ft..a  for  (2,  n  —  2)  degrees  of  free¬ 
dom  in  Table  A-5. 

(4)  Choose  a  number  of  values  of  X  (within  the 
range  of  the  data)  at  which  to  compute 
points  for  drawing  the  confidence  band. 


(5)  At  each  selected  value  of  X,  compute: 

Yc  =  Y  +  &!  (X  -  X) 
and 

(6)  A  (1  —  a)  confidence  band  for  the  whole 
line  is  determined  by 

y,  ±  Wt  . 

(7)  To  draw  the  line  and  its  confidence  band, 
plot  Yc  at  two  of  the  extreme  selected 
values  of  X.  Connect  the  two  points  by  a 
straight  line.  At  each  selected  value  of  X, 
also  plot  Yc  +  Wt  and  Ye  —  Con¬ 
nect  the  upper  series  of  points,  and  the 
lower  series  of  points,  by  smooth  curves. 

If  more  points  are  needed  for  drawing  the  curves 
for  the  band,  note  that,  because  of  symmetry 
the  calculation  of  Wj  at  n  values  of  X  actually 
gives  at  2n  values  of  X. 


(3)  n  =  11 

Fm  (2,  9)  =  4.26 


(4)  Let:  X  =  200 

X  =  250 
X  =  300 
X  =  350 
X  =  400, 
for  example. 

(5)  See  Table  5-6  for  a  convenient  computa¬ 
tional  arrangement  and  the  example  cal¬ 
culations. 


(6)  See  Table  5-6. 


(7)  See  Figure  5-13. 


For  example:  Wi  (but  not  Ye)  has  the  same 
value  at  X  =  250  (i.e.,  X  —  39.73)  as  at 
X  =  329.5  (i.e.,  X  +  39.73). 
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TABLE  5-6.  COMPUTATIONAL  ARRANGEMENT  FOR  PROCEDURE  5-5.2.2.1 


X 

(X-X) 

Yc 

1  ,  (X-X)> 
n  1  S„ 

*vt 

W* 

Yc+  W, 

m 

200 

-89.73 

181.3 

0.384179 

268.12 

16.37 

47.8 

229.1 

133.5 

250 

-39.73 

214.4 

0.148404 

103.57 

10.18 

29.7 

244.1 

184.7 

300 

+  10.27 

247.5 

0.094751 

66.127  1 

8.132 

23.7 

271.2 

223.8 

350 

60.27 

280.6 

0.223219 

155.79 

12.48 

36.4 

317.0 

244.2 

400 

110.27 

313.7 

0.533810 

372.55 

1 

19.30 

56.3 

370.0 

| 

257.4 

X  =  289.727 
Y  =  240.727 


s]r  =  697.9074 

-  =  0.0909091 
n 

6,  =  0.661873 
S„  =  27454.182 


Y.  =  Y  +  6,  (X  -  X) 


4, 


-  4  [~ 


+ 


(X  -  Xy~ 


a/2 F  =  V&52  =  2.919 


W ,  «  V2F  s„. 


5-5. 2. 2.2  Give  a  (1  —  a)  Confidence  Inlerval  For  a  Single  Point  On  the  Line,  i.e.,  the  Mean  Value 
of  V  Corresponding  To  a  Chosen  Value  of  X  (X'). 


Procedure 


Example 


(1)  Choose  the  desired  confidence  level,  1  —  a 

(2)  Obtain  sK  from  Basic  Worksheet. 

(3)  Look  up  !i_a/2  for  n  —  2  degrees  of  freedom 
in  Table  A-4. 

(4)  Choose  X' ,  the  value  of  X  at  which  we  want 
to  make  an  interval  estimate  of  the  mean 
value  of  7. 

(5)  Compute: 

and 

Yt  =  Y  +  6,  (X'  -  X) 

(6)  A  (1  —  a)  confidence  interval  estimate  for 
the  mean  value  of  Y  corresponding  to 
X  =  X'  is  given  by 

Y  +  6,  (X  -  X)  db  W, 

=  yc±w2  . 


(1)  Let:  1  -  a  =  .95 

a  =  .05 

(2)  From  Worksheet  5-5.2 

Sy  =  26.418 

(3)  n  =  11 
t.m  for  9  d.f.  =  2.262 


(4)  Let 


(5) 


X'  =  300, 
for  example. 


W2  =  (2.262)  (26.418)  (0.3078) 

-  18.4 

Ye  =  247.5 

(6)  A  95%  confidence  interval  estimate  for  the 
mean  value  of  Y  at  X  =  300  is  the  interval 
247.5  ±  18.4,  i.e.,  the  interval  from  229.1 
to  265.9  . 
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5-5. 2. 2.3  Give  a  (1  —  a)  Confidence  Interval  Estimate  Fora  Single  (Future)  Value  of  V  Corresponding 
To  a  Chosen  Value  of  X  =  X'. 


Procedure 

(1)  Choose  the  desired  confidence  level,  1  —  a 

(2)  Obtain  s>-  from  Worksheet. 

(3)  Look  up  ti_/j  for  n  —  2  degrees  of  freedom 
in  Table  A-4. 

(4)  Choose  X',  the  value  of  X  at  which  we  want 
to  make  an  interval  estimate  of  a  single 
value  of  y. 

(5)  Compute: 

and 

y,  =  y  +  bt  (xr  -  x) 

(6)  A  (1  —  a)  confidence  interval  estimate  for 
Y'  (the  single  value  of  Y  corresponding  to 
X’)  is  given  by 

Y  +  ft  (X1  -  X)  ±  W, 

-  y,  ±  wt  . 


Example 


(1) 

Let:  1  « 

-  .95 

« 

-  .05 

(2) 

From  Worksheet  5-5.2 

Sr 

=  26.418 

(3) 

t.m  for  9  d.f. 

=  2.262 

(4) 

Let  X' 

=  300, 

for 

example. 

(5) 

W,  =  (2.262)  (26.418)  (1.0463) 

=  62.5 

Yc  =  247.5 

(6)  A  95%  confidence  interval  estimate  for  Y 
at  X  =  300  is  the  interval  247.5  ±  62.5, 
i.e.,  the  interval  from  185.0  to  310.0  . 


5-S.2.3  What  Is  tha  Confidence  Interval  Estimate  for  ft ,  the  Slope  of  the  True  Line,  Yx  —  ft  +  ftX? 


(1) 

Procedure 

Choose  the  desired  confidence  level,  1  —  a 

(1) 

Example 

Let:  1  -  a  =  .95 
a  -  .05 

(2) 

Look  up  L_«,'2  for  n  —  2  degrees  of  freedom 
in  Table  A-4. 

(2) 

n  =  11 

t.m  for  9  d.f.  =  2.262 

(3) 

Obtain  s*,  from  Worksheet. 

(3) 

From  Worksheet  5-5.2 
=  0.159439 

(4) 

Compute 

(4) 

Wt  —  s», 

Wt  =  2.262  (0.159439) 

=  0.360651 

(5) 

A  (1  —  a)  confidence  interval  estimate  for 
ft  is 

(5) 

6,  =  0.661873 

W,  -  0.360651 

6,  ±  W<  . 

A  95%  confidence  interval  estimate  for  ft 
is  the  interval  0.661873  ±  0.360651,  i.e., 
the  interval  from  0.301222  to  1.022524  . 
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CHAPTER  6 

POLYNOMIAL  AND  MULTIVARIABLE  RELATIONSHIPS 
ANALYSIS  BY  THE  METHOD  OF  LEAST  SQUARES 

6-1  INTRODUCTION 


-In  this  Chapter,  we  give  methods  for  estimating  the  coefficients  of,  and  for  answering  various 
questions  about,  multivariable  functional  relationships  of  the  form 

y  —  0o  x<>  +  0i  jci  -f-  .  .  .  -f  0*_i  i  (6-1) 

between  a  dependent  variable  y  and  a  number  of  independent  variables  x0,  xu  .  . . ,  zt_i.  We  restrict 
our  discussion,  however,  to  the  case  in  which  the  values  of  the  independent  variables  x0,  xit 
are  known  exactly,  and  errors  of  measurement  affect  only  the  observed  values  Y  of  y,  that  is,  to 
many-variable  analogs  of  the  FI  functional  relationships  considered  in  Paragraphs  5-3.1  and  5-4.1. 

Methods  for  the  analysis  of  many-variable  relationships  in  which  errors  of  measurement  affect  the 
values  of  the  x’s  involved  as  well  as  the  observed  Y’s,  i.e.,  the  multivariable  analogs  of  the  FII 
structural  relationships  considered  in  Paragraphs  5-3.1  and  5-4.3,  are  not  discussed  per  se  in  this 
Chapter.  If,  however,  the  errors  that  affect  the  z’s  are  not  errors  of  measurement,  but  rather  are 
errors  of  control  in  the  sense  of  Paragraph  5-4.3.2,  i.e.,  are  errors  made  in  attempting  to  set 

X*,  X,, .  .  .  ,  Xk-\,  equal  to  their  respective  nominal  values  x'a,  x[ . zi_i,  then  the  methods  of 

this  Chapter  are  applicable,  provided  that  the  errors  made  in  adjusting  Xa,  Xu  .  .  .  ,  X*_i,  to  their 
respective  nominal  values  are  mutually  independent  (or,  at  least,  are  uncorrelated). 

The  techniques  presented  in  this  Chapter  are  general.  They  are  applicable  whenever  we  know 
the  functional  form  of  the  relation  between  y  and  the  x’s,  and  are  primarily  concerned  with  esti¬ 
mating  the  unknown  values  of  the  coefficients  of  the  respective  terms  of  the  relationship.  Thus, 
taking  z0  =  1,  x,  =  x,  x2  =  x\  .  . . ,  x„  =  xm,  the  methods  of  this  Chapter  enable  us  to  estimate  the 
coefficients  of,  and  to  answer  various  questions  about,  an  mth  degree  polynomial  relationship 

y  —  0o  +  0#  +  02a;2  +  . . .  +  ftmXm  (6-2) 

between  a  dependent  variable  y  and  a  single  independent  variable  x.  Alternatively,  taking  x0  =  1, 
X\  =  x,  Xi  =  z,  x,  =  a;2,  z,  =  xz,  and  x4  =  z2,  the  techniques  of  this  Chapter  can  be  used  to  investi¬ 
gate  the  nature  of  a  quadratic  surface  relationship 

V  =  /30  +  (0#  +  ftz)  +  (ftz2  +  0#z  +  0#*)  (6-3) 

between  a  dependent  variable  y  and  two  independent  variables  x  and  z.  For  example,  we  may  wish 
to  test  the  hypothesis  that  the  surface  actually  is  a  plane,  i.e.,  that  0,,  0,,  and  04,  in  Equation  (6-3) 
are  equal  to  zero,  and  so  forth. 
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Multivariate*  statistical  relationships  analogous  to  the  SI  and  SI1  situations  considered  in 
Paragraphs  5-3.2  and  5-5.1  are  not  considered  per  se  in  this  Chapter.  If,  however,  Y  and  X„ 
Xi, .  .  .  ,  X*_i,  have  a  joint  multivariate  frequency  (probability)  distribution  in  some  definite 
population,  and  if  a  sample  of  size  n  is  drawn  from  this  population,  with  or  without -selection  or 
restrictions  on  the  values  of  the  X’a  but  without  selection  or  restriction  on  the  Y’s,  then  the  methods 
of  this  Chapter,  taking  X0  ^  1  throughout,  are  directly  applicable  to  estimating  the  coefficients 
of,  and  to  answering  various  questions  about,  the  multivariate  regression  of  7  on  Xt,  X2,  .  .  . ,  and 
X*_1(  namely, 

F( j|  =  0o  +  /3iXi  +  fi2X2  +  . . .  +  $t-iXk~i,  (6-4) 

where  Y (l)  is  shorthand  for  mr.x,x,  .  .  .  the  mean  value  of  all  of  the  Y’s  that  are  associated  in 
the  population  with  the  particular  indicated  combination  Xu  Xt . .  .  Xt-i,  of  values  of  the  X’s 
(see  footnote  of  Par.  5-3.2) — and,  where 

0o  =  mr  -  0i mXl  -  02 mx,  -  ...  -  $k~imXk  „  (6-5) 

m„,  .  .  . ,  mJt_„  are  the  population  means  of  Yi,  Xlt . . . ,  X*_i,  respectively.  The  fitted  regres¬ 

sion,  yielded  by  the  application  of  the  methods  of  this  Chapter  to  observational  data  of  this  kind, 
will  be  of  the  form 

Y (t|  =  bo  +J)iXi  -f  b2X2  (6-6) 

with  60  =  Y  —  b2X2  -  b2X2  -  ...  -  (6-7) 

where  Y,  Xu  X2, .  .  .  ,  Xk-U  are  the  means  of  Y,  X,,  X2,  .  .  .  ,  and  X*_„  in  the  sample;  and  each  b 

will  be  a  best  (i.e.,  minimum  variance  unbiased)  estimate  of  the  corresponding  true  0. 

When,  as  in  all  of  the  previously  mentioned  situations,  the  relationship  between  y  and  the  x’s  is 
linear  in  the  coefficients  whose  values  are  to  be  determined  from  the  data  in  hand,  the  Method  of 
Least  Squares  is  the  most  generally  accepted  procedure  for  estimating  the  unknown  values  of  the 
coefficients,  and  for  answering  questions  about  the  relationship  as  a  whole.  A  widely  applicable 
Least  Squares  Theorem  is  given  in  Paragraph  6-2 ;  and  its  application  to  a  general  linear  situation 
is  presented  in  detail  in  Paragraph  6-3,  with  worked  examples.  Special'applications  to  polynomial 
and  other  situations  are  discussed  in  subsequent  paragraphs  of  this  Chapter. 

The  numerical  calculations  required  for  least-squares  analysis  of  multivariable  relationships 
often  are  lengthy  and  tedious.  Hence,  this  Chapter  is  directed  toward  arrangement  of  the  work 
for  automatic  computation  on  modern  electronic  computers.  Consequently,  basic  equations  called 
for  in  the  calculations  are  written  both  in  traditional  and  in  matrix  forms.  This  Chapter  concludes 
with  a  discussion  of  matrix  operations  that  are  useful  both  in  formulating  and  in  carrying  out  the 
requisite  calculations,  Paragraph  6-9. 

In  most  instances,  related  Procedures  and  Examples  appear  on  facing  pages  in  this  Chapter. 


*  The  important  distinction  in  statistical  work  between  a  variable  and  a  variate  is  drawn  in  the  Kendall-Buckiand 
Dictionary  of  Statistical  Term*w  as  follows: 

Variable — Generally,  any  quantity  which  varies.  More  precisely,  a  variable  in  the  mathematical  sense,  i.e.,  a 
quantity  which  may  take  any  one  of  a  specified  set  of  values.  It  is  convenient  to  apply  the  same  word  to  denote 
non-measurable  characteristics,  e.g.,  “sex”  is  a  variable  in  this  sense,  since  any  human  individual  may  take  one  of 
two  “values",  male  or  female. 

It  is  useful,  but  far  from  being  the  general  practice,  to  distinguish  between  a  variable  as  so  defined  and  a  random 
variable  or  variate  (q.v.). 

Variate — In  contradistinction  to  a  variable  (q.v.)  a  variate  is  a  quantity  which  may  take  any  of  the  values  of  a 
specified  set  with  a  specified  relative  frequency  or  probability.  The  variate  is  therefore  often  known  as  a  random 
variable.  It  is  to  be  regarded  as  defined,  not  merely  by  a  set  of  permissible  values  like  an  ordinary  mathematical 
variable,  but  by  an  associated  frequency  (probability)  function  expressing  how  often  those  values  appear  in  the 
situation  under  discussion. 
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6-2  LEAST  SQUARES  THEOREM 

If  the  n  measurements  Yu  Y,,  .  . . ,  Y„  are  statistically  independent  with  common  variance  a2 
and  have  expected  values  E{Y ,-), 

E(Y{)  =  poXci  4-  PiZu  +  4- ...  4-  pk-iXk-i.i 

E{Yt)  =  fioZc  4-  fliXu  4-  fiiXti  +  . .  ■  +  0k-iXk~i,i  (6-8) 

E(Yn)  —  /3cXon  4  filZln  +  0lZln  4“  •  ■  •  4"  0k-lXk-i,n 

then  the  best  linear  unbiased  estimates  $ir . . .  ,  /§t_„  of  the  unknown  coefficients  are  given  by 

the  solution  of  k  simultaneous  equations,  called  the  normal  equations, 

ft  +  &  2  XcXi  +  .  . .  +  0k— i  2  XqfZk~\  ~  2  Xo  Y 
2  XiXq  4-  $i  2  xj  4-  . . .  +  2  X\Xk~\  =  2  X\Y  (6-9) 

&,  2  x*_iXo  4-  @x  2  Xk-iXi  -f-  .  .  .  4-  flk-i  2  x*_i  =  2  Xk-i  Y 
where  the  summation  is  over  all  of  the  n  values  of  the  variables  involved;  e.g., 

n 

2  x,z,  =  23  ZuXu, 

i-l 

and  the  estimate  of  a2  is  given  by 

S2  —  -  T  2j  [Y<  —  (0<#O.  +  filZli  +  •  ■  ■  4- 

n-k  t  (6-10) 


If  no  unique  solution  to  Equation  (6-9)  exists  (which  will  occur  when  one  or  more  of  the  x’s  are 
linearly  dependent,  for  example,  if  x,  =  ax2  4  6x,),  then  not  all  k  coefficients  can  be  estimated 
from  the  data.  Variables  may  be  deleted  or  several  variables  may  be  replaced  by  a  linear  function 
of  those  variables  so  that  a  solvable  system  involving  fewer  equations  results. 

In  situations  where  the  variance  of  the  Y’s  is  not  the  same  for  all  Y’s  and/or  there  is  correlation 
among  the  Y’s,  a  transformation  of  variables  is  required.  The  methods  for  these  cases  are  discussed 
later  in  this  Chapter. 

This  theorem  can  be  restated  using  matrix  notation  as  follows: 

Let,  Y  =  r  Yi  1,  X  *=  f  xM  Xu  . . .  x*_i,i  ~],  and  0 

Yj  _  Xtn  Xu  .  .  .  X*_m 

_  Y„  __  _  X|)„  X|„  ...  Xk—l  _ 

The  expected  values  of  the  Y’s  then  is  expressed  as 

E(Y)  =  X0,  (6-8M) 

and  the  condition  of  independence  and  common  variance  is  expressed  by 
Var  (Y)  =  V  -  r2I. 

Under  these  conditions,  the  minimum  variance  unbiased  estimates  0  of  0  are  given  by  the  solution 
of  the  normal  equations 

X'X0  =  X'Y.  (6-9M) 
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The  estimate  of  a1  is  given  by 

s2  =  «y  -  x$y  <y  - 

-  ^ZTk  (y'y  ~  £'*'y)- 


(6-10M) 


Equations  (6-8),  (6-9),  and  (6-10)  are  given  in  the  usual  algebraic  notation,  and  the  corresponding 
equations  in  matrix  notation  are  (6-8M),  (6-9M),  and  (6-10M). 


6-3  MULTIVARIABLE  FUNCTIONAL  RELATIONSHIPS 

6-3.1  USE  AND  ASSUMPTIONS 

Least-squares  methods  for  estimating  the  coefficients  of  a  functional  relation  of  the  form 

y  =  PfflCa  +  ftXi  +  +  .  .  .  +  ^k~lXt-x  (6-1) 

are  used  in  a  number  of  situations: 

(a)  when  it  is  known  from  theoretical  considerations  in  the  subject  matter  field  that  the  rela¬ 
tionship  is  of  this  form; 

(b)  when  the  exact  expression  relating  y  and  the  x's  either  is  unknown  or  is  too  complicated 
to  be  used  directly  and  it  is  assumed  that  an  approximation  of  this  type  will  be  satisfactory. 

In  the  latter  case,  the  approximation  often  can  be  justified  on  the  grounds  that,  for  the  limited 
range  of  the  x’s  considered,  the  surface  representing  y  as  a  function  of  the  x’s  is  very  nearly  the 
hyperplane  given  by  Equation  (6-1).  The  method  is  strictly  valid  in  (a),  but  in  (b)  there  is  danger 
of  obtaining  misleading  results,  analogous  to  the  bias  arising  in  the  straight-line  case  from  the 
assumption  that  the  functional  relation  involved  is  linear  when  in  fact  it  is  not  linear. 

In  addition  to  the  validity  of  Equation  (6-1),  the  following  assumptions  must  be  satisfied:* 

(a)  the  random  errors  in  the  Y's  have  mean  zero  and  a  common  variance  a2; 

(b)  the  random  errors  in  the  T’s  are  mutually  independent  in  the  statistical  sense. 

For  strict  validity  of  the  usual  tests  of  significance,  and  confidence  interval  estimation  procedures 
in  Paragraph  6-3.3  (Steps  8  and  9),  an  additional  assumption  must  be  satisfied: 

(c)  the  random  errors  affecting  the  T’s  are  normally  distributed. 

The  x  variables  may  be  powers  or  other  functions  of  some  basic  variables,  and  several  different 
functions  of  the  same  x  variable  may  be  used.  (See,  for  example,  Equation  (6-2)  or  (6-3)). 

The  data  for  analysis  consist  of  the  n  points  (xm,  xu, .  . . ,  x*_i.i,  Y j)  (x02,  x12,  .  .  . ,  x*_i,s  Fj) .  . .  , 
(x0n,  xu, ....  Tn),  and  usually  are  represented  in  tabular  form  as: 


X „ 

Xx 

X , 

• 

• 

♦ 

Y 

Xoi 

Xn 

Xu 

Xt-1.1 

Yx 

Xn 

Xu 

Xk- 1 ,» 

Yt 

Xo» 

X,n 

Xu 

Xk-l.n 

Yn 

*  When  these  assumptions  are  not  satisfied,  see  Paragraph  6-6  for  the  case  of  inequality  of  variance,  and  Paragraph  6-7 
for  the  case  of  correlation  among  the  P’s. 
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Alternatively,  the  data  may  be  expressed  in  the  form  of  observational  equations, 

00*01  +  01*11  +  02*21  +  •  •  •  +  0*-l**-l,l  =  Yl  —  Vl  +  €l 
00*02  +  01*12  +  02*23  +  •  ■  -  +  0*-l**-l.l  ~  Yj  =  +  e, 

(6-11) 


0O*On  +  01*1-  +  03*2-  +  .  •  -  +  0*-l**-l,«  =  Y»  =  J/-  +  e- 

where  eu  eJt . . . ,  e„  denote  the  errors  of  the  Y’s  as  measured  values  of  the  corresponding  true  y’s. 
When  the  number  of  observational  equations  exceeds  the  number  of.  unknown  coefficients,  i.e., 
when  n  >  k,  the  observational  equations  ordinarily  are  mutually  contradictory;  that  is,  the  values 
of  0o,  0i, ... ,  and  0t_i  implied  by  any  chosen  solvable  selection  of  k  of  the  equations  do  not  satisfy 
one  or  more  of  the  remaining  n  —  k  equations.  Hence,  there  is  a  need  for  best  estimates  of  the  0’s 
based  on  the  data  as  a  whole. 

For  a  unique  least-squares  solution,  n  must  not  be  less  than  k,  and  the  normal  equations  (6-9) 
must  be  uniquely  solvable.  If  not,  some  variables  must  be  deleted  or  suitably  combined  with  other 
variables. 

6-3.2  DISCUSSION  OF  PROCEDURES  AND  EXAMPLES 

In  setting  forth  the  steps  in  the  solution,  the  formulas  are  given  in  the  usual  algebraic  notation 
and  also  in  matrix  notation  where  appropriate. 

Data  Sample  6-3.2,  selected  for  arithmetical  simplicity,  serves  to  illustrate  the  worked  examples 
of  numerical  procedures  involved  in  estimating  the  coefficients  of,  and  in  answering  various  questions 
about,  multivariable  functional  relationships. 

Data  Sample  6-3.2 


*1 


1 

2 

2 

3 

4 
4 


*2 

8 

8 

6 

1 

2 

5 


*» 


1 

7 

0 

2 

7 

1 


Y 

2 

4 

4 

4 

3 

3 


We  assume  that  these  data  correspond  to  a  situation  in  which  the  functional  dependence  of  y  on 
*i,  Xt,  and  *i,  is  of  the  form 

V  =  01*!  +  02*2  +  0»*s,  (6-12) 

which  is  a  special  case  of  Equation  (6-1)  with  the  term  0o*o  omitted;  i.e.,  with  0«  taken  equal  to 
zero.  Equation  (6-12)  implies  that  the  functional  dependence  of  y  on  xu  *2,  and  *,,  takes  the  form 
of  a  hyperplane*  that  passes  through  the  origin  (0,  0,  0,  0)  of  the  four-dimensional  Euclidean  space 


•  A  flat  surface  in  four  or  more  dimensions  is  termed  a  hyperplane  when  it  is  the  locus  of  points  that  vary  in  more 
than  two  dimensions. 
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whose  coordinates  are  xh  xs,  and  y.  If  we  wished  to  allow  for  the  possibility  that  the  dependence 
of  y  on  xu  Xi,  and  x»,  may  take  the  form  of  a  hyperplane  that  intersects  the  y-axia  at  some  point 
(0,  0,  0,  ft),  not  necessarily  the  origin,  then  we  would  substitute 

y  *  ftx0  +  ftxj  +  ftXj  +  ftzj 

for  Equation  (6-12),  and  take  x0  =  1;  i.e.,  amend  Data  Sample  6-3.2  by  adding  an  x0  column  of  l’s. 

By  analogy  with  Equations  (6-11),  Data  Sample  6-3.2  and  the  assumed  functional  relationship 
Equation  (6-12)  can  be  summarized  symbolically  by  observational  equations  of  the  form 

ftXn  4  ftXji  -f-  ftx,i  =  Y i 

ft*n  +  ftXjj  +  ftXn  .  —  Yi 

(6-13) 


ftxu  +  ftX2#  +  ft^M  =  Y« 


Substitution  of  the  values  of  the  x’s  and  Y’s  of  Data  Sample  6-3.2  in  Equation  (6-13)  gives 

ft*l  +  ft-8  +  ft-.l  =  2 
ft*2  +  ft-8  +  ft-7  =  4 

ft-2  +  ft-6  +  ft-0  =  4  (6-14) 

ft -3  +  ft-l  +  ft-2  =  4 
ft-4  +  ft-2  +  ft-7  =  3 
Pi'  4  +  ft  •  5  +  ft  •  1  =3 

as  the  observational  equations  corresponding  to  Data  Sample  6-3.2. 


6-3.3  PROCEDURES  AND  EXAMPLES 

Step  1  Procedure — Formation  of  Normal  Equation*.  The  normal  equations  are  formed  from  the 
sums  of  squares  and  cross  products  as  follows: 

ft2xj|  +  ftSzoXi  +  .  .  .  +  ft_,2xoX*_,  =  2xoY 

ft2x,x0  +  ft2x?  +  . . .  -f  ft_i2x,x*^i  =  2Xi  Y  (6-9) 


ft2x*_,x0  +  ft2xt_!X,  +  ■  -  -  +  ft_i2xj_i  =  2x*_1Y 
or  in  matrix  form 

X'Xp  =  X'Y  =  Q  (6-9M) 

where  Q'  =  (qu  q2, . . .  ,  qr), 

and  9,  =  2  xa  Y< ,  O'  =.0,  1 . k  -  1). 
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Step  1  Example — Formation  of  Normal  Equations.  The  normal  equations  (See  Equations  (6-9)) 
corresponding  to  the  observational  equations  (6-13)  are 


ft  2Z*  +  ft  2Z,Z2  +  ft  2Z,Z3 
Pi  2z,za  +  ft  2zJ  +  ft  2z2z3 
Pi  2XiX,  +  ft  Zz2za  +  ft 


or  in  matrix  form 


X'  X  p  =  X' Y 


where 


p  = 


ft 

ft 

’ft. 


2^,7 

ZzsY 

2z3Y 


(6-15) 


(6-15M) 


Numerical  evaluation  of  the  requisite  sums  of  squares  and  sums  of  cross  products  for  Data  Sample 
6-3.2  and  substitution  in  Equation  (6-15),  yields 


ft-50  +  ft-67  ■+■  ft *53  =54 
ft -67  -f-  ft  >194  +  ft  >85  =  97 
ft  >53  -f-  ft -85  +  ft  >104  =  62 

and  the  matrices  involved  in  Equation  (6-15M)  become 


(X'X)  = 

2zf 

2z,z2 

2z,Zj 

r=  1 

"50 

67 

53" 

2z,z2 

2z^ 

Zz2z3 

67 

194 

85 

_2z,Zj 

2z2z3 

_53 

85 

104 

"2z,y' 

= 

■54" 

= 

V 

zz2y 

97 

ft 

_2z3  y_ 

_62_ 

-ft_ 

(6-16) 


(6-16M) 
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St«p  2  Procedure— Solution  of  Normal  Equations.  Equations  (6-9)  can  be  solved  by  a  number  of 
methods  giving  values  for  which  can  be  expressed  as 

$o  •=  c ooQo  +  CoiQt  +  .  .  .  +  Ca,k-iQk~i 

&  “  Cio9o  +  cnq\  +  . . .  +  (6-17) 

$*-l  =  Ck-i.oQo  +  Ck~ i,tQi  +  .  .  •  +  Ck-\,k-\<lk~\ . 

A  solution  for  the  /§/ s  can  be  arrived  at  without  explicitly  computing  the  c,/s,  of  course,  but  in 
the  following  computations  the  c,/s  are  needed.  The  values  of  the  c</s  depend  only  on  the  sums 
of  squares  and  cross  products  of  the  independent  variables  Zg,  xu  . . . ,  xk,  so  that  the  estimates  of 
the  fix’s  can  be  expressed  as  a  linear  function  of  the  Y’s. 


In  matrix  notation,  this  step  is  given  by  computing  the  inverse  of  the  matrix  of  normal  equations, 
i.e., 


(X'X)-' 


=5 

Cm 

Coi 

.  .  .  Cg,k- 1 

Cio 

Cn 

Ci,k-l 

_C*-1 .0 

Ck- l.i 

and  Equations  (6-17)  become 


fi  -  (X'X)~lX'Y  (6-17M) 

=  ( X'X )~*Q. 
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Step  Z  Exampl«— S«lMtton  of  Nonna)  Equation*.  The  values  k,  k,  and  k,  that  constitute  the 
solutions  of  the  normal  equations  can  be  expressed  (See  Equations  (6-17))  in  the  form 

k  =  cn  (/i  4-  Ch  qx  4-  c„ 

=  Cn-54  4-  Cu*97  -f-  c,j •  62 

j§j  =  Cn  Qi  +  Cn  qx  +  c„  q,  (6- 18) 

=  Cn •  54  4*  Ctj'97  +  Ct>*62 
j§»  Cji  qi  +  Cn  fli  +  c* a  qt 
=  Cn*54  Cn' 97  4-  c»t*62 


where  the  c’s  are  the  elements  of  the  inverse  matrix 


(X'X)~« 


Cl! 

Cn 

Cn 

Cn 

Crt 

c„ 

Cn 

Cm. 

( X'X)_l  may  be  computed  in  many  ways.* 


The  exact  inverse  of  the  matrix  (X'X)  determined  by 


the  first  equation  of  Equations  (6-16M)  is 

'  12951  -2463 

(X'X)->  - 


239418 


-2463 

-4587 


2391 

-699 


-4587" 

-699 

5211 


(6-18M) 


where  the  factor  in  front  of  the  matrix  is  to  be  applied  to  the  individual  terms  in  the  matrix. 


Using  the  first  equation  of  Equations  (6-18),  we  get 


k  =  23^8  ,(12951)  (B4)  +  {~2m)  (97)  +  ("4687)  (62)1 

*  ^5418  *  699364  ~  238911  ”  284394) 

=  176049 
239418 

=  0.735  320  652. 


The  other  coefficients  are  obtained  similarly: 

k  -  0.232  175  526 
k  =  0.031  664  286 


The  prediction  equation,  therefore,  is 
f  =  0.735  320  652  x,  4-  0.232  175  526  x,  4-  0.031  664  286 


*  The  advent  of  automatic  electronic  digital  computers  has  reduced  the  inversion  of  matrices  of  even  moderate  sixe 
to  a  matter  of  seconds.  Routines  for  matrix  inversion  are  standard  tools  of  automatic  computation.  In  contrast, 
matrix  inversion  by  desk  calculators  is  a  time-consuming  and  tedious  affair.  Detailed  illustration  at  this  juncture  of 
any  one  of  the  common  methods  of  matrix  inversion  by  desk  calculator  would  not  only  constitute  a  distinctive  interrup¬ 
tion  to  the  orderly  presentation  of  the  essential  features  of  this  Chapter,  but  would  lengthen  it  considerably.  The  two 
most  common  methods  of  matrix  inversion  by  desk  calculator — the  Doolittle  method,  ana  the  abbreviated  Doolittle  method 
(also  called  the  Gaues-DooliUle  method) — are  described  and  illustrated  by  numerical  examples  in  various  statistical 
textbooks,  e.g.,  in  Chapter  15  of  Anderson  and  Bancroft.1*1  Details  of  the  square-root  method,  favored  by  some  com¬ 
puters,  are  given,  with  a  numerical  illustration,  in  Appendix  11A  of  O.  L.  Davies’  book/’1  Ail  of  the  common  methods 
of  matrix  inversion  by  desk  calculators  are  described  in  considerable  detail,  illustrated  by  numerical  examples,  and 
compared  with  respect  to  advantages  and  disadvantages  in  a  paper  by  L.  Fox,  Practical  Solution  of  Linear  Equation* 
and  Inversion  of  Matrices,  included  in  Taussky,1"  Reference  also  may  be  made  to  the  book  of  Dwyer.'"  The  reader 
of  this  Handbook  who  is  faced  with  matrix  inversion  by  desk  calculator  is  referred  to  these  standard  sources  for  guidance 
and  details. 
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Step  3  Procedure— Calculation  of  Deviation  Between  Predicted  and  Observed  Value  of  the  Vs. 

The  predicted  value  Y,  at  a  given  point  (z0„  xu, ....  Yd  is  given  by  substituting  the  values 
of  x  in  the  prediction  equation,  i.e., 

Y,  -  0oXa<  -I-  /3i*n  +  & Xu  +  •  •  .  +  0k-lXk-l.il 

and  the  residuals  r,  -  Y,  —  Y,  are  given  by 

Ti  =  Yi  —  Yj  =  Yj  —  (j3&X<u  4"  fan  4*  •  .  .  +  0k-iXk-i.i) 

r2  =  Yi  —  Y{  =  Y2  —  (/9o£(>2  +  P1X12  4"  •  •  •  4"  $k-lXk-l,l) 

(6-19) 


rn  —  Yn  Yn  —  Y„  (rfcXon  4“  ^i^i»  4“  ...  4“  0k— iXk— 1,*) 
or  in  matrix  notation 

r  =  Y  —  X$  (6-19M) 

where 


r  1 

Ti 


T  = 


In  classical  least-squares  analysis,  Y,  is  termed  the  adjusted  value  of  the  observed  value  Y,.  It  is 
important  to  distinguish  between  the  errors  of  the  Y<  with  respect  to  the  corresponding  true  values 
y„  and  the  residuals  of  the  Y,  with  respect  to  their  adjusted  or  predicted  values  Y,- ;  that  is,  between 
the  e<  of  Equations  (6-11)  and  the  r,  of  Equations  (6-19). 


Stop  4  Procedure— Estimation  of  a2.  The  estimate  s’  of  o7  is  computed  from 


or  in  matrix  notation 

•*  -  rh (rV) 


(6-20) 


(6-20M) 
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Step  3  Example — Calculation  of  Deviation  Between  Predicted  and  Observed  Value  of  the  Vt.  The 

predicted  or  adjusted  values  f ,  corresponding  to  the  observations  Yt  are  obtained  by  substituting 
the  values  of  the  x’s  into  the  prediction  equation.  For  the  first  observation,  substituting  xtl  -  1, 
xn  =  8,  Xji  -  1  leads  to 

=  0.735  320  652  (l)  +  0.232  175  526  (8)  +  0.031  664  286  (1) 

=  2.624  389  146. 

The  corresponding  residual  is 

r,  =  Yt  —  tx 
-  2  -  2.624  389  146 
=  -.624  389  146. 

The  full  data,  the  corresponding  predicted  values  (?,•)  and  their  residuals  (r<),  are: 


i 

Xu 

Xu 

Xu 

y. 

?< 

Residuals 

r< 

1 

1 

.  8 

1 

2 

2.624  389  146 

-.624  389  146 

2 

2 

8 

7 

4 

3.549  695  514 

.450  304  486 

3 

2 

6 

0 

4 

2.863  694  460 

1.136  305  540 

4 

3 

1 

2 

4 

2.501  466  054 

1.498  533  946 

5 

4 

2 

7 

3 

3.627  283  662 

-.627  283  662 

6 

4 

5 

1 

3 

4.133  824  524 

-1.133  824  524 

Stop  4  Example — Estimation  of  <r*.  The  estimate  81  of  <rs  may  be  computed  directly  from  the  sum 
of  squared  residuals.  Thus, 


=  ^  (5.808  473  047) 

=  1.936  157  682 

where  n  is  the  number  of  observational  points  (here  6)  and  k  is  the  number  of  coefficients  estimated 
from  the  data  (here  3).  Alternatively,  s*  may  be  evaluated  from 

•'-rbj*1” 

=  g~g  {70  -  (0.735  320  652)  (54)  -  (0.232  175  526)  (97)  -  (0.031  664  286)  (62)} 

=  |  (5.808  473  038) 

=  1.936  157  679. 

Extracting  the  square  root  gives 
s  =  1.391  4588. 
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i. 


Step  5  Procedure— SsHmotad  Standard  Deviations  of  Hie  Coefficients.  The  estimated  standard 
deviation  of  4,  is  given  by  sV c«,  where  Cu  is  the  s'th  diagonal  term  of  the  inverse  of  the  matrix  of 
normal  equations 

est.  s.d.  of  &  ■*  sVcoo 
est.  s.d.  of  &  =  sVc^ 
est.  s.d.  of  &  =■  sV^ 

(6-21) 


est.  s.d.  of  $k-i  **  sVc*_i.*_i. 


Step  6  Procedure— Standard  Deviation  of  a  Linear  Function  of  the  fit.  The  standard  deviation  of 
-  L  **  ao$0  +  aji  +'at$t  +  ...  +  is -estimated  by 

est.  s.d.  of  L  «=*  8+  y]  Y]  o,o, j  (6-22) 

\  J-O 

or  in  matrix  form 

est.  s.d.  of  L  -  sVf'fX'j'Q-'f  (6-22M) 

where  V  =  (o^.a,.  . . .  ,  a*_i). 

Case3  of  special  interest  are: 

(a)  estimate  of  a  single  coefficient,  i.e.,  L  =  in  which  case  Equation  (6-22)  reduces  to  Equa¬ 
tion  (6-21); 

(b)  estimate  of  the  difference  of  two  coefficients,  i.e.,  t  *  /§,  —  $„  in  which  case  Equation  (6-22) 
becomes 

est.  s.d.  of  0i  -  /§,)  -  sVc^r+^TT-V^i  ■  (6-23) 


Stop  7  Procedure— Standard  Deviation  of  a  Predicted  Point.  Using  the  results  of  Step  6,  the  pre¬ 
dicted  yield  fh,  at  any  chosen  point  (xfl*,  xlh, . .  . ,  z*_i,„),  is  given  by 

?h  =  0oXqK  +  $lXlh  +  .  .  .  +  4*^1, X*_I  ,A 

which  is  a  linear  function  of  the  p's.  Application  of  Equation  (6-22)  leads  to 

est.  s.d.  of  fh  =  «Jg  S  (6-24) 

V  »-o  j-o 

or  in  matrix  notation 

est.  s.d.  of  t k  =  sVl'(X'X)-H 
where  f'  =  (xM,  xu, . .  . ,  x*_ljA) 


(6-24M) 
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Stop  5  Example — Estimated  Standard  Deviations  of  the  Coefficients.  The  values  of  the  estimated 

standard  deviations  of  the  #’s  are: 


Coefiicient 

Vui 

Estimated 
Standard  Error  of 
Coefficient,  *Vc« 

A 

.232  581 

.323  627 

A 

.099  934 

.139  064 

A 

.147  631 

.205  283 

Stop  6  Example— Standard  Deviation  of  a  Linear  Function  of  the  Coefficients.  For  illustrative  pur¬ 
poses,  consider  L  -  &  —  10  /§,. 

By  Equation  (6-22),  or  Equations  (6-22M), 
est.  s.d.  of  L  =  a y/ +  100  c„  —  20  cu 
in  matrix  notation 

est.  s.d.  of  L  -  aVl'(X'X)-'l 
with  V  =  (0,  1,  -10). 

Numerical  evaluation  in  this  instance  gives 

est.  s.d.  of  L  -  1.391  4688  ‘ 

-  2.0848. 


Stop  7  Example— Standard  Deviation  of  a  Predicted -Point.  By  Equation  (6-24),  or  Equation 
(6-24M),  the  estimated  standard  deviation  of  the  predicted  yield  corresponding  to  an^  chosen 
point  (xik,  Xtk,  Xih),  is  given  by 

est.  s.d.  of  fk  =  Zj  ^ 

or  in  matrix  notation 

-  aVWX'X)-H 

where  V  =  (xu,  a;,*). 

Thus,  the  estimated  standard  error  of  the  predicted  or  adjusted  yield  corresponding  to  the 
first  observational  point  (1,  8,  1),  is 

est.  s.d.  of  P,  -  «(c»  4-  8cu  +  c„  +  8c, j  +  64cw  +  8c„  +  c,i  +  8c, ,  -f  c*]» 

- 1891  4688  (gas)' 

-  0.949  235. 
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Step  8  Procedure — Analytic  of  Variance  Test  of  Significance  of  a  Group  of  p  <  k  of  the  Coefficient*. 

To  test  the  statistical  significance  of  a  set  of  p  of  the  0’s  (for  simplicity  the  last  p),  start  with  a 
reduced  set  pf  normal  equations,  omitting  the  last  p  rows  and  columns,  and  repeat  Steps  2,  3,  and  4, 
as  a  problem  with  ( k  —  p)  variables: 

(a)  The  equations  in  Step  2  then  are  reduced  to 

2a$)o  +  .  .  .  +  ZX&k-p-A-p-i  =  Qo 


(6-25) 


2*«a5t_p_i^o  +  . . .  +  Zzi-p-A-F-i  = 

and  its  solution  becomes 

ft  =  CooQo  +  •  •  ■  +  C*  .t-P-lQk-p-l 

(6-26) 

0Jfc  —p—\  —  Ci  ,k—p~ iQo  ”f"  •  ■  •  "f"  C)c—p—l,k—p—iQk—p—i‘ 

(b)  These  values  c*  will,  in  general,  be  different  from  the  c„  for  the  original  equations,  so  that 
new  coefficients 

&•  a*  6*  a* 

Po  <  Pi  *  Pa  •  •  •  •  »  Pk  -p-l 
will  result. 

(c)  A  new  value  of  s2,  say  s*\  is  computed  from 

-  n-(Lp)  jSy’  -  *?'  »  «<j  <6-27> 

These  operations  can  be  handled  conveniently  by  matrix  methods.  Paragraph  6-9  contains  a 
further  discussion  of  “Matrix  Methods.” 

An  Analysis  of  Variance  table  is  formed  as  follows: 


d.f. 

Sum  of  Squares 

Mean  Square 

Total 

n 

ZY 2 

Reduction  due  to  k  constants 

k 

23  Qi 

0 

K 

Residual  (after  k  constants) 

n  —  k 

2YJ  -  2  /§•  q. 

0 

s * 

Reduction  due  to  A:  —  p  constants 

k  —  p 

*2?  ft  9. 

€ 

A 

Residuals  after  k  —  p  constants 

n  -  (k  -  p) 

ZY2  -  ‘2l  ft  <7. 

0 

8*' 

Reduction  due  to  additional  p  constants 

P 

*-l  _  *“P~1  . 

23 hq*-  ±  ft* 

o  o 

t  P 

If  the  y’ s  are  normally  distributed  about  their  expected  values,  then 
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Step  8  Example— Analysis  of  Variance  Test  of  Significance  of  Last  Coefficient.  The  required 

Analysis  of  Variance  table  is: 


d.f. 

Sums  of  Squares 

Mean  Square 

Total 

6 

70.000  000 

Reduction  due  to  3  constants  (&,  ft,  and  $$) 

3 

64.191  527 

21.397  176  =  K 

Residuals  (after  3  constants) 

3 

5.808  473 

1.936  158  -  s» 

Reduction  due  to  ft  and  ft  only 

2 

64.145  461 

32.072  730  =  A 

Residuals  (after  ft  and  &) 

4 

5.854  539 

1.463  635  =  s ♦* 

Reduction  due  to  ft 

1 

.046  066 

.046  066  »  P 

As  implied  by  Equation  (6-27),  the  sum  of  squares  for  the  reduction  due  to  &  and  ft  only  = 
<7i  +  0*  9s,  where  $*  and  fit  are  the  estimates  of  ft  and  ft  that  are  obtained  when  ft  is  taken  equal 
to  zero;  i.e.,  when  the  underlying  functional  relation  is  taken  to  be  y  -  fitXi  +  ft**. 


The  steps  required  to  evaluate  jlf  and  are 

(*'*)  =  I"60  671 

(A,A,)  ^g7  lg4j 

<*'*>"  '  MI  [-67  "£] 

wyi  -  [“]  *  [fl 

[jC]  =  (X'X)-'  [X[Y] 

_  __1_  r  194  -  671  !" 64l 

5211  L- 67  50JL97J. 

They  yield 

ff  =  0.763  193  245 
A*  =  0.236  422  951. 


Hence,  reduction  due  to  0*  and  fit  only  is  given  by 
(0.763  193  245)  (54)  +  (0.236  422  951)  (97)  =  64.145  461 
as  shown  in  the  Analysis  of  Variance  table. 


6-15 


AMCP  706-110 


ANALYSIS  OF  MEASUREMENT  DATA 


Stop  •  Procedure  (Cent) 

v 

(a)  F  =  —  is  distributed  as  F  with  di.  «  k,n  —  k,  and  serves  as  a  test  of  whether  all  k  constants 
account  for  a  significant  reduction  in  the  error  variance. 

p 

(b)  F  -  —  is  distributed  as  F  with  d.f.  ■»  p,  »  —  k,  and  serves  as  a  test  of  whether  the  addition 

of  the  p  coefficients  accounts  for  a  significant  reduction  in  the  error  variance  over  that  accounted 
for  by  the  first  k  —  p  constants. 

NOTE:  In  cases  where  a  constant  term  is  involved  (i.e.,  Xoi  ■=  1)  we  would  use 

js  Qt  ~  ^jp-|/(*  ~  1) 

F  "  «* 

which  is  distributed  as  F  with  (k  —  1)  and  (n  —  k)  degrees  of  freedom  as  a  test  for  the  efficacyof  the 
prediction  equation. 


Stop  9  Procedure — Confidence  Interval  Estimates.  Li  and  L t  constitute  a  100  (1  —  a)  %  confidence 

interval  estimate  for: 

(a)  a  coefficient  0,-, 

when  Li  =  (est.  s.d.  of  $,) 

Li  =  (est.  s.d.  of  &); 

(b)  a  predicted  point  on  the  curve  ?it 

when  Li  =  ?,•  —  !,_*.«  (est.  s.d.  of  ?,) 

Lt  =  +  <»-*..  (est.  s.d.  of  ?,); 

(c)  a  difference  of  two  coefficients 

when  1^  =  (0.  -  (est.  s.d.  of  -  0,) 

Li  =  0>  -  $i)  +  (est.  s.d.  of  -  $j). 

In  the  above,  is  the  value  of  Student's  t  for  (n  —  k)  degrees  of  freedom  exceeded  with 
probability 
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Slap  8  Examp!*  (Conf) 

The  test  of  significance  for  $t  is 


■046  060 
=  1.936  158 

-  .024,  di.  -  1,  3. 

The  value  of  F(  1,  3)  exceeded  with  probability  .05  is  10.13.  The  observed  F  does  not  exceed 
this  critical  value,  so  that  fit  is  not  regarded  as  being  statistically  significantly  different  from  zero. 


Stop  9  Examp!* — Confkfonc*  Interval  Estimates.  For  fit,  the  95%  confidence  interval  estimate 
Li  <  fit  <  Li  is  determined  by 

In  -  /9i  —  t,.  m  (est.  s.d.  of  $i) 

=  0.735  320  652  -  3.182  (.323  627) 

=  -.294  460 

In  —  $i  +  tt..m  (est.  s-d.  of  $t) 

=  0.736  320  652  +  3.182  (.323  627) 

=  1.765  102 

where  U.  oi  —  3 . 182  is  the  value  of  Student’s  t  distribution  for  three  degrees  of  freedom  exceeded 
with  probability  .025  (or  exceeded  in  absolute  value  with  probability  .05). 


6-4  MULTIPLE  MEASUREMENTS  AT  ONE  OR  MORE  POINTS 

More  than  one  measurement  may  be  made  at  some  or  at  all  of  the  values  of  the  independent 
variable  x.  This  usually  is  done  when  the  random  errors  are  suspected  of  being  composed  of  two 
components — one  component  associated  with  the  variation  of  the  points  about  the  curve,  and  the 
other  component  associated  with  the  variation  of  repeat  determinations.  The  /th  measurement  at 
the  tth  point  then  can  be  represented  as 

Y a  “  fivX^i  +  fiiZu  +■*•.+■  fik—iXt — i . *  4-  ti  +  (6-28) 

where  the  e's  and  ij’s  are  independent  and  have  variances  a*  and  <rj,  respectively. 

If  a  number  p(  of  repeat  determinations  are  made  at  each  of  the  »  points,  the  estimation  of  o'1 
and  <rj  follows  from  a  modification  of  the  Analysis  of  Variance  table: 
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Sum  of  Squares 

d.f. 

Mean  Square 

Total 

x;x;n  -  t 

±P< 

1 

i-l  /-I 

Reduction  due  to  fitted  constants 

S^(2x<y)  =  C 
»-  0 

k 

C/k 

Residual  (after  fitted  constants) 

T  -  C  =  R 

2 Pi  -  k 

R/VPi  -  k) 

Repeat  determinations 

?.)*  =  E» 

i-l  i-l 

Xpi  —  n 

Ea/{Zpi  -  n) 

Variations  of  averages  about  the  curve 

g? 

ii 

ca 

i 

QJ 

n  —  k 

Ei/(n  -  k) 

The  expected  value  of  E0/(Xp,  —  n)  is  <r”,  and  the  expected  value  of  E,/(n  —  k)  is  a\  +  pa2, 
where  all  the  pt  are  equal  to  p. 

The  quantity  [Ei/(n  —  k)]/{Ea/ (Zp,  —  n)]  =  F  is  (under  the  assumption  of  a  normal  distribution 
for  the  t  s  and  t]  s)  distributed  as  F,  if  a1  =0,  with  n  —  k  and  Zp*  —  n  degrees  of  freedom,  and  may 

be  used  to  test  the  statistical  significance  of  the  component  of  variance  associated  with  the  «’s  by 
comparing  the  observed  F  value  with  tables  of  the  F  distribution. 

If  all  the  pi  are  equal,  the  proper  variance  estimate  to  use  in  calculating  the  standard  errors,  or 
confidence  intervals  of  the  estimated  constants,  is  E,/(n  -  Jfc). 


6-5  POLYNOMIAL  FITTING 

If  it  can  be  assumed  that  the  relation  between  the  dependent  variable  Y  and  the  independent 
variable  x  is 

Yi  =  /So  +  PiXi  +  prf  +  ...  +  &_,x*-1  +  (6-29) 

and  that  the  errors  of  measurement  are  independent  and  have  the  same  variance  a2,  then  the 
techniques  for  multiple  regression  carry  over  without  change,  by  setting: 

Xoi  —  1,  Xu  —  Xf,  Xjt  =  Xi  f  .  .  .  f  X*_i ™  xf  . 

The  normal  equations  are 

n/30  +  ZX&  +  2xJ&  +  ...  +  Zx*-1^  =  ZT 

ZxA,  +  Zx’fr  +  Zx*02  +  -  . .  +  2x*0t_,  =  ZxY  (6-30) 

2x*->0o  +  Zx*^i  +  Zx*+*&  +  . .  .  +  Zx2*-*/?*.,  -  Zx*-lT. 

Note  that  if  the  constant  term  is  assumed  to  be  zero,  variable  x0  is  dropped,  and  the  first  row  and 
column  are  dropped  from  the  normal  equations. 
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In  using  a  polynomial  as  an  approximation  to  some  unknown  function,  or  as  an  interpolation 
formula,  the  correct  degree  for  the  polynomial  usually  is  not  known.  The  following  procedure 
usually  is  applied: 

(a)  Carry  through  the  steps  in  fitting  polynomials  of  2nd,  3rd,  4th,  5th,  .  .  .  ,  degrees. 

(b)  If  the  reduction  in  the  error  sum  of  squares  due  to  fitting  the  Ath  degree  term  is  statistically 
significant  on  the  basis  of  the  F-test,  whereas  the  similar  test  for  the  (k  +  1)  degree  term  is  not, 
then  the  Ath  degree  polynomial  is  accepted  as  the  best  fitting  polynomial. 

In  this  procedure,  the  degree  of  the  polynomial  is  a  random  variable,  and  repetitions  of  the  experi¬ 
ment  will  lead  to  different  degree  polynomials.  When  the  law  is  truly  polynomial,  the  computed 
curve  will  either  be  of  correct  degree  and  hence  will  give  unbiased  estimates  of  the  coefficients  or, 
if  at  of  correct  degree,  will  lead  to  biased  estimates. 

When  the  law  is  not  exactly  a  polynomial,  the  error  distribution  for  the  Y's  will  be  centered 
around  a  value  off  the  curve,  and  it  will  be  difficult  to  assess  the  effect  of  such  systematic  errors. 
In  the  limiting  case,  where  the  variance  of  the  Y’s  is  nearly  zero,  these  systematic  errors  will  be 
treated  as  the  random  error  in  the  measurements.  Usually,  it  will  not  be  valid  to  assume  that  these 
systematic  errors  are  uncorrelated.  On  the  other  hand,  if  these  systematic  errora  are  small  relative 
to  the  measurement  error,  their  effect  probably  can  be  neglected. 


6-6  INEQUALITY  OF  VARIANCE 

6-6.1  DISCUSSION  OF  PROCEDURES  AND  EXAMPLES 

When  the  measurements  Y,  have  different  precision,  i.e.,  when  V(Y<)  =  <A  and  <r„  ^  <r<,  for  at 
least  one  pair  of  subscripts  1  <  ii  <  it  <  n,  the  conditions  of  the  least  squares  theorem  of  Para¬ 
graph  6-2  are  not  satisfied.  However,  the  transformed  variates 


<Ti 


have  a  common  variance  P(Y,')  =  1.  Often,  we  have  information  on  the  relative  magnitudes  of 
the  variances  <rj  only,  and  not  on  their  absolute  magnitudes.  If  the  variances  a\  are  expressed 
in  the  form 


then  w,  is  termed  the  relative  weight*  of  the  measurement  Y„  and  the  quantities  Y?  =  y/w,  Y, 
have  common  variance  <j\,  the  magnitude  of  which  may  be  unknown.  In  other  words,  equality  of 
variance  is  achieved  through  weighting  the  observations  by  quantities  proportional  to  the  reciprocals 
of  their  standard  deviations. 


*  The  aftsolufe  weight  of  a  measurement  is,  by  definition,  the  reciprocal  of  its  variance. 


6-19 


AMCP  706-110 


ANALYSIS  OF  MEASUREMENT  DATA 


6-6.2  PROCEDURES  AND  EXAMP1ES 
Procedures — The  equations  representing  the  expected  values  of  the  Y*  are 

E(Jf)  -  Vw,E(Yd 

=  fitVm  Xn  +  AVt 7f  zu  +  ...  +  A-iVwi  Zk-i.t  (6-32) 

«=  A  xj  +  A  *5  +  • .  -  +  Pk-i 
where  x*  —  Vrc*  z,> 

The  normal  equations  for  the  estimation  of  the  0's  are 

+  *«*iA  +  ...+Z*  ZtXk-i0k-i  =  Sw  x»Y 

2  w  XoXjA  -f  Im  XjA  + _ +2w  =  2  «?  XiY  (6-33) 

2  w  ZoXit-i0o  +  2  w  XiXt-10,  +  ...  +  J#  4-iA-i  *  2  w  x*_i  Y 


The  estimate  s1  of  »£  is  given  by  the  usual  formula 

±Yr  -£&,(±Z$  Y? 

i-i  J-1  \  i-1 

st  « - ^ - 

n  —  k 

which  may  be  written,  in  terms  of  the  original  variables,  as 

Y) 

n  -k 


2  w.-YJ  -  2  4 


'(!>• 


(6-34) 


(6-35) 


Note  that  in  the  case  where  the  value  of  <rj  is  known,  we  may  perform  a  test  of  significance  of 

gl 

the  closeness  of  the  observed  estimate  to  the  known  value  by  forming  the  ratio  F  =  -j  and 
comparing  this  value  with  the  100  (1  -  a)  percentage  point  of  the  F  distribution  for  n  —  k  and  <x> 

(ft 

degrees  of  freedom;  or,  equivalently,  we  may  compare  x*  =  - - -r* —  with  the  10O-(l  —  a) 

percentage  point  of  the  x1  distribution  for  n  —  Jfc  degrees  of  freedom.  Restatement  of  the  foregoing, 
using  matrix  notation,  goes  as  follows: 

If  Var  (Y.)  -  Diag  <r*J 

=  *?Diag  ), 

\W|  Wt  wnj’ 

then  the  transformed  variates 


Y*  =  Diag  (Vw i  Vwj . . .  Vw^) Y  =*=  WY  and 
X*  =*  Diag  (Vwi  Vwj .  . .  VwjX  *  WX 

satisfy  the  requirements  of  the  least  squares  theorem  of  Paragraph  6-2,  and  the  normal  equations  are 


(X*)'(X*)0  -  (X*)'Y*  or, 
X’  W*  X  $  =  X'  W*  Y. 


(6-33M) 
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The  estimate  of  aj  is  given  by 

s’  =  |Y*'  Y*  -  4'  or, 

n  —  K 

.  t'  W ’  r 

s’  « - r 

n  —  k 


Example* — Fitting  Straight  Lin*  Relation  (Variance  of  Y  Proportional  to  Abscissa). 

estimation  of  the  coefficients  of  a  line  where 

V  **  a  +  fiX\, 

and  where  Var(Y<)  =  <r*x„  t  -  1,  2, . . . ,  n.  The  equations  of  expectation  are 

E(Y{)  =*  a  4-  fix i 
E(Yt)  =  a  +  fix x 

E(Yn)  —  a  +  fixn 


Transforming  to  Y*  =  Y./Va,  gives 

E(Y  t)  =  -2=  +  0V5; 

Vli 

®07)  =  +  fiVix 

E(Y*)  =  -7-  +  /9V£ 

VX, 

and  the  normal  equations  for  estimating  a  and  fi  become 
<82^  +  $n  -  2  Y</x, 

Xi 

<8n  +  /92x=*2Y, 


Direct  solution  of  these  equations  gives 


n  2  —  -  2  Y, 

U1-) 

Xi 

\  x<) 

a  =  w  2  Yj  -  2x,  2Y,/x.- 

»’  -  <’*■> 1  (j) 
and  for  the  estimate  of  <r’, 

8 


(6-84M) 

Consider  the 

(6-86) 

(6-37) 

(6-88) 

(6-39) 

(6-40) 

(6-41) 
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6-7  CORRELATED  MEASUREMENT  ERRORS 


6-7.1  DISCUSSION  OF  PROCEDURES  AND  EXAMPLES 

If  the  errors  of  measurement  are  not  independent  but  instead  are  correlated  so  that  they  have 
covariances 

Covar  (Y<,Y*)  -  a ti  =  (6-42) 

and  variances 

Var  (Y<)  =  4, 

then  a  transformation  of  the  variables  Yu  Yt,  ,  Y„,  to  new  variables  Yf,  Y*t  . .  .  ,  Y*,  is 
required  so  that  the  method  of  least  squares  may  be  applied.  In  some  simple  cases,  a  transformation 
in  the  form  of  sums  and  differences  of  the  original  variables  immediately  suggests  itself,  and  the 
expected  values  of  the  new  variables  are  computed  easily.  The  example  used  to  illustrate  the 
techniques  presented  in  this  Paragraph  is  such  a  case. 


6-7.2  PROCEDURES  AND  EXAMPLES 

Procedures — The  variances  and  covariances  may  be  represented  by  the  n  x  n  variance-covariance 
matrix 


V  = 

<Tlj 

<ru 

ffll 

4 

<Tln 

_<rnl 

<r»* 

1 

(6-42M) 


Assuming  V  to  be  of  full  rank,  i.e.,  determinant  of  V  is  not  zero,  it  is  possible  to  factor  V  into  the 
product 


V  -  T  T'  (6-43M) 

where  T  is  lower  triangular  and  T’  is  the  transpose  of  T.  The  required  transformation  then  is 
given  by 


Y*  =  T~l  Y  and  X*  =  T~l  X  (6-44M) 

where  (Y*)'  =  (Y*,  Y*, .  . . ,  Y*)  is  the  vector  of  transformed  variables  and  Y'  =  (Yx,  Y„  . . . ,  Y„) 
is  the  vector  of  original  variables.  X*  and  X  are  the  matrices  representing  the  equations  of 
expected  values  of  the  transformed  variables  and  of  the  original  variables,  respectively.  (See 
Paragraph  6-9  for  the  method  of  computing  T  and  T~l). 


The  normal  equations  then  are 
(X*)'(X*)  j3  =  (X*)'  Y* 
or,  in  terms  of  the  original  variables, 
X'V~'X  $  =  X'Y->  Y, 
and  the  estimates  of  the  jS’s  are  given  by 

0  =  {(X*)'(X*))-“l(X*)'Y* 

-  (X'Vz'X)-'X'V-'Y. 

The  variance  estimate 


(6-45M) 

(6-46M) 

(6-47M) 


5* 


(6-48) 


6-22 


POLYNOMIAL  AND  MULTIVARIABLE  RELATIONSHIPS  AMCP  706-110 


Procedures  (Coni) 

is  an  estimate  of  unity  when  the  variances  and  covariances  are  known.  This  may  be  written  as 


s1  =  * — — r  {Y'V-»Y  -  $'X’V-'Y J 

7%  K 


($48M) 


where  r  is  the  column  vector  of  deviations,  r  =  Y  —  X$. 

If,  instead  of  V,  a  matrix  with  entries  proportional  to  the  variances  and  covariances  is  used,  say 

W  ='\v,  then  si  is  an  estimate  of  <rj. 

O'o 


Example*— Parabolic  Relationship  with  Cumulative  Error*.  If  the  errors  of  measurements  of  Y  at 
successive  x  values  in  a  case  of  a  parabolic  law  Y  **  ft  +  fts  +  ft#1  are  cumulative,  i.e., 

Yt  =  ft  +  ft*i  +  ft*!  +  Cl 
Y*  ft  +  ft*l  +  ft*J  +  «J  +  «J 


ft  +  ftZfi  -f  ftX*  + 


then  E{Y)  = 


xt 

Xt 


ft 

ft 

.ft 


-  Xft 


If  all  the  «'s  are  from  the  same  distribution, 

then  Var  (Y.)  =  i  •  a1 

Covar  (Y,-  Y,)  =  {*  *  <  j 

U  *  3  <  1 

and  the  variance  covariance  matrix  becomes 


Taking  W 


— ,  V,  the  necessary  transformation  is  given  by  factoring  W  into  W 


TT\ 
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6-7.2  PROCEDURES  AND  EXAMPLES  (CONT) 
Examples  (Cent) 

A  little  commutation  gives 


W 


1 

1  1 


1 


1  1 
1  1 
1 


1  1  1  .  .  .  1 


1 


W- 1  =  ( T')-'T~l 


1 


0  0 

-1  0 

1  -1 


L 


1 


0  0  1 
0  0-11 


2  -1 

-1  2  -1 

-1  2  -1 


-1 


2  -1 

-1  1 


which,  for  the  transformed  variate,  gives 


Hi 

• 

II 

-s 

h: 

ii 

~  l  " 

~Yt 

Yi 

-l  l 

Yt 

Yt  -  r, 

0  -1  1 

Y , 

y,  -  r, 

0  0  0  ...  -1  1_ 

Yn  -  y.-,_ 
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Examples  (Coni) 
and 


X*  -  T-'X  = 

1 

i  *1  A~ 

-1  1 

i  z,  A 

0-11 

i  z.  A 

.  0  0  0  ...  -1  1J 

.1  Z»  2*. 

1  x,  A 

0  x,  -  x,  A  ~  A 

0  Xi  —  x%  A  ~  A 


l_0  x.  -  Xn—i  A  ~  ai-ij. 

Note  that  y*  »  y,  -  y^, 

-  (x<  -  Xi-i)fo  +  (A  -  A-iMh  +  «<  for  *  >  2 
and  Y*  -  y,; 

hence,  the  Vb  have  the  same  variance,  and  have  zero  covariances. 


The  normal  equations  become 


1  Xi 

xi  A  +  X  (*<  -  *i- i)f 


*1  +  S  (*<  -  *<-i)  (*?  ~  4-i) 

s 


10 


A  +  1Z  (*.  -  Xi-i)  (A  -  *?_,) 


A  +  X)  (4  -  A-t)* 
1 


-y, 

*iy.  +  s  (*<  -  *<-i)  (y<  -  y*_,) 
*?y»  +  ±(A~  *5-,)  (y*  -  y^o 
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6-7.2  PROCEDURES  AND  EXAMPLES  (CONT) 
Examples  (Cont) 


or,  in  terms  of  the  original  matrices  X'W~lX0  ~  X'W~lY,  give 


1  1 

Xt  Xt 

LA  A 


x\ 


2  -1 

-1  2 
-1 


2  -1 


-1 


2 

•1 


•1 


Xt 

X, 


A' 

A 

A 


n  i 

Xi  X2 

A  A 


l 

X„ 

A 


2 

-1 


-1 

2 


-1 


-1 


riv 

Y , 

Y, 


L-f  n_ 


which,  upon  multiplication,  will  be  seen  to  give  the  same  normal  equations  as  above.  If  the  analysis 
is  carried  out  in  terms  of  the  transformed  variables,  <r*  is  estimated  by 


2 Y*‘  - 


**  -  (2  x*  y;j 

n  -  3 


or  equivalently,  in  terms  of  the  original  variables,  by 


8*  =  —-3  ( Y'W-'Y  -  fr'X’W-'Y) . 


6-8  USE  OF  ORTHOGONAL  POLYNOMIALS  WITH  EQUALLY  SPACED  *  VALUES 

6-8.1  DISCUSSION  OF  PROCEDURES  AND  EXAMPLES 

The  fitting  of  a  polynomial 

Y  —  +  ft x  +  +  . .  .  -f-  Pk-iXl~l  (6-49) 

to  observations  at  n  equally-spaced  values  of  x  (spaced  a  distance  D  apart)  can  be  simplified  by 
transforming  the  x’s  to  new  variables  C  which  are  orthogonal  to  each  other. 
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The  variables  then  become 
$o  =  {o  =  1 


x,  -  X 

b 


£1  =  >u£i  where  {, 

ft  ■=  X-jft  where  ft  =  -)  -  U—y£^' 

(i  -  x,ft  where  ft  -  (?-^)’  -  (“  jr)  (“ 55^)  (6-50) 

«  -  »here  *  -  (H-)'  ~  (-TT5)  +  560  <«‘  -  10»’  +  9> 

ft  _  x,ft  wheK  ft  .  (^)*  -  ^  +  (^) 


feJ  (n*  -  it*) 


where  £*+l  =  {,{*  -  4-^rTTj  &-«• 

The  X,  are  chosen  so  that  the  elements  of  £,  are  integers. 


By  fitting  Y  as  a  function 

Y  =*  oof'o  +  «i$l  4*  • .  •  +  ft- 1 .  (6-51) 

the  estimation  of  the  «’s  and  the  analysis  of  variance  are  simplified  because  the  normal  equations 
are  in  diagonal  form. 


In  order  to  obtain  the  estimates  of  the  /3’s  and  their  associated  standard  errors,  or  to  use  Equa¬ 
tion  (6-51)  for  predicting  a  value  for  a  point  not  in  the  original  data,  an  extra  calculation  but  no 
matrix  inversion  is  required. 


Tables  of  X,  and  Z(£')2  are  given  by  Fisher  and  Yates'6’  for  n  <  75,  and  by  Anderson  and 
Houseman'7’  for  n  <  104  for  up  to  5th  degree  polynomials;  in  DeLury(8)  for  n  <  26  for  all  powers; 
and  in  Pearson  and  Hartley'9’  for  n  <  52  for  up  to  6th  degree  polynomials.  Table  6-1  is  a  sample 
from  Fisher  and  Yates."1’ 


To  illustrate  the  calculations,  consider  the  fitting  of  a  cubic  to  the  following  (x,  Y)  points: 


X 

Y 

10 

3.4 

20 

11.7 

30 

37.2 

40 

80.1 

50 

151.4 

60 

253.2 

70 

392.6 
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TABU  4-1.  SAMPlf  TABU  Of  OtTHOOOMAL  POtTNOMIAU 


3 

f']  v» 

4 

*'»  Vi  V* 

5 

Vi  Vj  Vi  V * 

* 

£',  £'.  1'.  {'. 

7 

Vi  Vi  V *  £'*  | 

• 

t'l  Vi  Vi  Vi  V » 

-1  +1 

0  -2 
+  1  +  1 

-3  +1  -1 

-1  -1  +3 

+  1  -1  -3 

+3  +1  +1 

-2  +2  -1  +1 
-1  -1  +2  -4 

0-2  0+6 
+  1  -1  -2  -4 

+2  +2  +1  +1 

1 

-5  +5-5+1  1  i 

-3  -1  +7  -3  +5 

-1  -4  +4  +2  - 10  i 

+  1  -4  -4  +2  +10  i 

+3  -1  -7  -3  -5 

+  5  +5  +5  +1  +1 

-3  +5  -1  +3  -1 

|  -2  0+1  -7  +4 

-1  -3  i-1  +1  -5 

0  -4  0  +6  0 

+  1  -3  -1  +1  +5 

+  2  0-1  -7  -4 

+3  +5  +1  +3  +1 

-7  +7  -7  +7  -7 

-5  H  +6  -13  +23 
-3  -3  +7  -3-17 

-1  -5  +3  +9  -15 

+  1  -5  -3  +9  +15 

+  3  -3  -7  -3  +17 

+  5  +1  -5  -13  -23 
+7  +7  +7  +7  +7' 

Z<£T  2  6 

20  4  20 

10  14  10  70 

70  84  180  28  252 

28  84  6  154  84 

168  168  264  616  2184 

X  1  3 

2  1  V 

1  1  l  H 

2  1  |  It 

1  1  {  A  n 

2  1  1  A  A 

• 

£'l 

9 

V * 

£'< 

£'« 

£'. 

£'. 

10 

Vi 

£'. 

£'. 

{'■ 

11 

Vs  V, 

£'. 

{'■ 

£', 

11 

Vi 

£'• 

{'■ 

0 

-20 

0 

+  18 

0 

+  1 

-4 

-12 

+  18 

+6 

0 

-10  0 

+6 

0 

+  1 

-35 

-7 

+  28 

+20 

+i 

-17 

-9 

+9 

+  9 

+3 

-3 

-31 

+3 

+  11 

+  1 

-9  -14 

+4 

84 

*3 

-29 

-19 

+  12 

+  44 

+2 

-8 

-13 

-11 

+4 

+5 

-1 

-35 

-17 

+  1 

+2 

-6  -23 

-  1 

44 

4  5 

-17 

-25 

-13 

+  29 

+  3 

+  7 

-7 

-21 

-11 

+7 

+2 

-14 

-22 

-14 

4  3 

-I  -22 

-6 

-1 

47 

+i 

-21 

-33 

-21 

+  4 

+  28 

+  14 

+  14 

+4 

+  9 

+6 

+  42 

4 18 

+6 

M 

4  6  —6 

-6 

-6 

+  9 

4  26 

3 

-27 

-57 

+  5 

+  15  +30 

+6 

43 

+  11 

+  55 

+33 

+83 

+33 

(£')' 

60 

990 

468 

330 

8.680 

780 

110 

4,290 

156 

572 

5.148 

15,912 

2.772 

2,002 

132 

2,860 

858 

286 

12,012 

8,008 

X 

i 

3 

< 

A 

A 

2 

1 

) 

A 

1^ 

* 

1  t 

A 

A 

2 

3 

i 

A 

A 

— 

- . - 

— 

_ _ 

— 

... - 

— 

- 

- - 

. -  ... 

_ _  . 

. J 

— 

_ _ 

_ _ 

_ _ 

£'l  £'» 

13 

Vs 

£'< 

£'. 

e'i 

14 

Vi 

£4 

£'. 

£'. 

£'. 

13 

Vi 

£'. 

£'. 

0 

-14 

0 

+84 

0 

+  i 

-8 

-24 

+  108 

+  60 

0 

-56 

0 

+  756 

0 

+1 

-13 

-4 

+  64 

+20 

+3 

-7 

-67 

+  63 

+  145 

+1 

-53 

-27 

+  621 

+675 

+2 

-10 

-7 

+  11 

+26 

+  5 

-5 

-95 

-13 

+  139 

+2 

-44 

-49 

+261 

+  1000 

+3 

-5 

-8 

-54 

+  11 

+7 

-2 

-98 

-92 

+  28 

+  3 

-29 

-61 

-249 

+751 

+4 

+2 

—6 

-96 

-18 

+  9 

+2 

-66 

-132 

-132 

1  4-4 

-8 

-58 

-704 

-44 

+5 

4-11 

0 

-66 

-33 

+  11 

+7 

+  11 

-77 

-187 

1  +5 

+  19 

-35 

-869 

-979 

+6 

+  22 

+11 

+  99 

+  22- 

+  13 

+  13 

+  143 

+  143 

+  143 

+6 

+  52 

+  13 

-429 

-1144 

|  +7 

+  91 

+91 

+  1001 

+  1001 

<£')* 

182 

572 

6,188 

910 

97,240 

235,144 

1  280 

39,780 

10,581,480 

2,002 

68,068 

728 

136,136 

37,128 

6,466,460 

X 

1 

1 

» 

A 

rt# 

2 

l 

< 

A 

A  ; 

1 

3 

1 

ft 

It 
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TAIU  t-l.  1AMPU  TABU  Of  OBTHOSOKAL  KHYNOMIAU  (Cl 


•  •- — 

... 

- - -  - 

_  . 

. . . 

— 

- - 

tz 

10 

{ < 

v  9 

;  < 

L  « 

f . 

(  1 

Cl 

«'• 

•-  • 

Ci 

s’. 

i 

+i 

-21 

-63 

+  189 

+46 

0 

-24 

0 

+86 

0 

+1 

-40 

-8 

+44 

+220 

+s 

-19 

-179 

+  129 

+  115 

+  1 

-28 

-7 

+81 

+55 

+3 

-37 

-23 

+33 

+583 

+s 

-15 

-266 

+28 

+  131 

+2 

-20 

-13 

+  17 

+88 

+5 

-31 

-35 

+18 

+783 

+7 

-9 

-801 

-101 

+77 

+8 

-15 

-17 

-8 

+83 

+7 

-22 

-42 

-12 

+588 

+9 

-1 

-267 

-201 

-88 

+4 

-8 

-18 

-24 

+86 

+9 

-10 

-42 

-36 

+  166 

+11 

+9 

-148 

-221 

-143 

+  5 

+  1 

-15 

-39 

-39 

+  11 

+5 

-33 

-51 

-429 

+  18 

+21 

+91 

-91 

-148 

+6 

+  12 

-7 

-89 

-104 

+  18 

+28 

-13 

-47 

-871 

+  15 

+86 

+456 

+278 

+  148 

+7 

+25 

+7 

-18 

-91 

+16 

+44 

+20 

-12 

-676 

+8 

+40 

+28 

+52 

+  104 

+  17 

+  88 

+68 

+68 

+884 

x  <n* 

1,890 

1,007,760 

201,662 

408 

3,876 

100,776 

1,938 

28,266 

6,953,644 

6,712 

470,288 

7,762 

16,796 

28,266 

28,424 

X 

2 

1 

V 

A 

A 

1 

1 

4 

A 

A 

2 

1 

4 

A 

A 

No**:  Dor  a  >  8,  ooir  the  nhn  far  tawlttee  (\  =  --jj-  >n 

Note:  In  lUh  II,  tub  A*  nln  far  mUn  &  —  *  j*  arc  fhti  for  1>I  TW  mbdlf  wlw  (a ft  ram  far  amn  and  (a  —  l)/t 

■W*  far  a  odd)  mat  br  rejjplfad  hr  mtag  the  store  row*  hi  mm  order,  chaasfe*  tbo  tiro  for  odd-aaaibend  t.  See  a  =  7  and 
«  =  l  far  ooaiill  of  tbfo  rolr. 


fa*  afc  CSSal  »a*  Stlii 


*a«*i.a*.A.m,  m  r.  To* 


From  Table  6-1,  for  n  «  7  we  copy  out: 


£ 

*; 

e; 

Y 

1 

-3 

5 

-1 

3.4 

1 

-2 

0 

1 

11.7 

1 

-1 

-3 

1 

37.2 

1 

0 

-4 

0 

80.1 

1 

1 

-3 

-l 

151.4 

1 

2 

0 

-l 

253.2 

1 

3 

5 

i 

392.6 

where 

a  =  i 

z  —  X 


«  = 


10 

«  -4 


with  t  =*  40,  D  =  10. 


X*  =  1 
X,  -  1 
X,  ~  1 
X,  -  1/6 
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j. 


6-9.2  PROCEDURES  AND  EXAMPLES 

Stop  1  Procedure — Form  the  quantities 

XY 

2 

(6-52) 


Sti -,Y 


and,  using  the  values  of  2£*»v,  ...»  given  in  Table  6-1  form  the  estimates  of  the 
parameters,  a0,  au  . . . ,  as  follows: 


4> 


XY 

n 


4 


s 


(6-58) 


W-iY 

2{iii 


Stop  2  Procodur* —  Calculate  the  deviations  r,  from 

r<  =  Yi  —  fl  —  <Si  (i,i  —  djfJ.j  —  ...  —  (6-54) 
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Step  1  Example — Using  the  values  copied  from  Table  6-1,  the  following  calculations  are  made: 


2  fJY  =  929.6  2*;*  =  7 

2*;y  =  1764.8  2f|*  =  28 

2  £Y  =  1093.8  2?;*  =  84 

2  ?tY  =  33.5  2{J*  =  6. 


The  estimates  of  the  coefficients  in  the  representation  of  y  as  a  function  of  the  i.e.,  as 
V  =  ao{o  +  «iil  +  +  ai& 


are  given  by 

(So  =  2&Y/2&  =  929.6/7  =  132.8 
<$,  =  2{,T/2?;>  =  1764.8/28  =  63.0285  7143 
<S,  =  2&Y/2&  =  1093.8/84  =  13.0214  2857 
<s,  =  2?;y/2f;j  =  33.5/6  =  5.5833  3333. 


Step  2  Example — The  predicted  value  for  the  point  x  *=  10  is  given  by  substituting  its  corre¬ 
sponding  values  of  the  ?'s  ($'  =  1,  {(  =  —3,  £  «=  5,  and  £  =«  —1)  in  the  equation 

fx  =  132.8  +  63.028  5714  +  13.021  4286  £  +  5.583  3333  £, 

i.e.,  fia  -  132.8  +  63.028  5714(-3)  +  13.021  4286(5)  +  5.583  3333(-l) 

-  3.238  0955 

leading  to  a  deviation  between  observed  and  calculated  of 

r10  =  3.4  -  3.238  0955 
=  0.161  9045. 

For  the  entire  set  of  points,  we  get: 


Observed 

y 

Calculated 

? 

Residual 
r  -  y  -  ? 

3.4 

3.238  0955 

0.161  9045 

11.7 

12.326  1905 

-0.626  1905 

37.2 

36.290  4761 

0.909  5239 

80.1 

80.714  2856 

-0.614  2856 

151.4 

151.180  9523 

0.219  0477 

253.2 

253.273  8095 

-0.073  8095 

392.6 

392.576  1905 

0.023  8095 
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Stop  3  Procedure — The  estimate  of  9 1  is  given  by 

«*  =  {zy*  -  d&Y  -  -  ...  -  4*-i2^,yj .  (6-66) 


Stop  4  Procedure — The  estimate  of  the  standard  deviations  of  the  d*s  is  given  by 

s.d.  (d,)  -  ~f=-  (6-66) 

vS  (3 

Y  *-l 


Stop  5  Procedure — The  Analysis  of  Variance  table  becomes: 


d.f. 

Sum  of  Squares 

Total 

n 

zy* 

Reduction  due  to  fitting  a* 

1 

A  (zy)  -  R* 

Deviations  from  fit  with  <*> 

»  -  1 

zy*  -  Rt 

Reduction  due  to  fitting  at 

1 

*  (Z«y>  -  Rt 

Deviations  from  fit  with  <*,  a, 

i»  —  2 

zy*  -  R,-  ft 

Reduction  due  to  fit  of  a*_i 

1 

d*_i  (z£_,y)  *■  Rt- 1 

Deviations  from  fit  with  a*  alt _ _  «*_t 

n-k 

zy.  —  R*  —  R\ ...  —  Rt- 1 
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Step  3  Example — The  estimate  of  «r*  is  given  by 


=  |  (1.676  9048) 

=  .658  9683 
s  =  V.568  9683 
=  .7476. 


Step  4  Example — The  standard  deviations  of  the  coefficients  are  given  by 
s.d.  («S.)  =  s/VW' 

s.d.  (<*<,)  =  .7476/ V7  =  .2826' 

s.d.  (<«,)  =  .7476/V28  «  .1413 

s.d.  (*,)  =  .7476/>/84  =  .0816 

s.d.  (<S,)  =  .7476/V6  «  .3062. 


Step  5  Example — The  Analysis  of  Variance  table  becomes: 


d.f. 

Sum  of  Squares 

Mean  Square 

Total 

7 

249  116.26 

Reduction  due  to  coef.  of  & 

1 

123  450.88 

123  450.88 

Residuals  from  /S^ 

6 

125  664.38 

20  944.06 

Reduction  due  to  coef.  of 

1 

111  232.822  86 

111  232.82 

Residuals  from  <S0£j  + 

5 

14  431.557  14 

2  886.31 

Reduction  due  to  coef.  of  fj 

1 

14  242.838  57 

14  242.84 

Residuals  from  +  <Sj(j 

4 

188.718  57 

47.18 

Reduction  due  to  coef.  of 

1 

187.041  67 

187.04 

Residuals  from  <S„£j  +  . .  .  + 

3 

1.676  90 

.5590 
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Sfep  6  Procedure — Convert  to  an  equation  in  the  original  x  units  by  substituting  the  expressions 
in  Equations  (6-50)  into  Equation  (6-51).  By  writing  the  /§’s  as  linear  functions  of  the  <S's,  say 

$k  =  <$<, 

i—0 

the  standard  deviation  can  be  computed  from 
s.d.  of  (s-d-  «*.)’■ 


for  polynomials  up  to  5th 
terms  involving  if  3rd 


given  by 


and  the  s.d.  of  is  estimated  by 


See  Equations  (6-57)  on  page  6-36. 


The  following  Equations  (6-57)  show  the  $’s  as  a  function  of  the  <$’s 
degree.  (If  a  polynomial  of  4th  degree  is  used,  simply  disregard  the 
degree,  disregard  the  terms  involving  cS4  and  <S5;  etc.) 

As  an  example,  if  a  4th  degree  polynomial  is  fitted,  the  estimate  is 
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Step  6  Example — To  obtain  the  equation  in  terms  of  the  original  x  variable,  i.e.,  expressing  y  as 
y  —  fa  +  pix  +  +  PtX1,  we  substitute  as  follows: 

v  =  «0  (i)  +  «x  (tt)  +  [(?ir)  - 4]  +?  [( - 7  (Hr5)] 


(«0 


4a,  +  12«2  6a,)  +  (1(j  io  a’  +  60  “*)  X  +  (lOO  10o)  **  +  6000  **' 


Substituting  the  estimated  values  for  the  a’s  gives 


Y  =  3.4428  5714  -  .299  007  9375  x  +  .018  547  6191  x *  +  .000  930  5556  x*. 


The  standard  deviations  of  the  jS’s  are  given  by 
s.d.  of  $0  =  s.d.  of  ((So  —  4(5!  -)-  12<Ss  —  6<5») 


_  „/l  (  — 4)J  (12)*  (-6)* 

\7  +  28  +  + 


84 


59 


7 

=  2.170 


,  d  0f  a  -  *  /( 6)’  ,  (-48)*  (41)* 

s.d.  of  fix  -  60  V  28  +  g4  +  6 


=  .2190 


S.d.  Of  &  -  ^  + 


/I  ,  (-2)* 
100  \84  +  6 


.006  158 


B.d.  of 


8  1 
“  6000  a/6 

=  .0000  5087. 
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6-9  MATRIX  METHODS 

6-9.1  FORMULAS  USING  TRIANGULAR  FACTORIZATION  Of  NORMAL  EQUATIONS 

The  matrix  for  the  left-hand  aide  of  normal  equations  can  be  factored  into  (X'X)  ~  TT'  where 
T  is  lower  triangular,  so  that  {X'X)~l  —  ( T ')  1  1  =  ( T~ ■)'  T~\ 

Thus,  $  =  (T-*)'  (T-«Q)  where  Q  -  X'Y. 

Denote  the  column  vector  by 

g  =  T~Hj  =  iv 

JDk_  • 

Therefore,  $  —  (T~l)'g. 

This  representation  leads  to  certain  simplifications,  e.g.: 

(a)  The  estimate  of  <rJ  is  given  by 

«’  =  ^Tk  (yT  ~ 

=  ~-k  (Y'Y  -  ff'T-K?) 


(b)  The  variance  of  a  linear  function,  L  —  o'(§  of  the  $'s  is  given  by 


s’  fa'  (T-')'  T~l  a}  =  s1  (T-«o)'  CT-'a) 
=  s*  ZA’ 


(c)  The  reduction  in  sum  of  squares  due  to  fitting  the  last  p  constants  is 


2  0l. 


k-p+t 
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This  formulation  also  permits  us  to  make  a  detailed  Analysis  of  Variance  table.  An  important 
caution  is  in  order.  The  reduction  due  to  the  addition  of  /§,■  is  the  reduction  given  that  4x»  #*, . . . , 
have  been  fitted  to  the  data.  The  reduction  due  to  given  that  any  other  set  of  coefficients  have 
been  fitted  will  be  different. 

The  Analysis  of  Variance  table  becomes: 

d.f. 

Sum  of  Squares 

Reduction  due  to  fitting 

1 

A 

Residual  (after  fitting 

n  —  1 

27*  -  g\ 

Additional  reduction  fitting 

1 

ft 

Reduction  due  to  fitting  &  and  & 

2 

A  +  ti 

Residual  (after  fitting  &  and  ft) 

n  —  2 

zyt  -  gf  -  Ql 

Additional  reduction  due  to  fitting 

1 

ffl 

Reduction  due  to  fitting  &,  & ,  ...»  4* 

k 

±1?? 

X 

Residual  (after  fitting  $u  &,...,  $*) 

n  —  k 

27*  -  ±g\ 

1 

This  form  of  analysis  is  especially  useful  in  the  analysis  for  polynomials  where  the  ordering  is  by  ' — S 

powers  of  x.  In  the  multiple  regression  case,  the  reduction  attributed  to  /§,•  is  dependent  upon  the 
ordering  of  the  parameters  ft,  ft,  ...  ,  u  and  will  be  different  for  different  orders. 

6-9.2  TRIANGULARIZATION  OF  MATRICES 

The  real  symmetric  matrix 


Oil 

On 

■ 

fll," 

Oil 

Ojj 

Oj, 

tt»i 

O.J 

can,  if  N  is  non-singular  (i.e.,  if  |2V|  ^  0),  be  factored  into  the  product  of  two  triangular  matrices 
so  that  N  =  TT',  i.e., 


All 

On 

. 

Oi, 

= 

Cn 

“C.i 

Cn 

. 

C„1 

Oil 

Ojj 

. 

Oi, 

Cu 

Cj! 

c*  i 

Cnl 

_o„i 

0,1 

o„_ 

Jnl 

c«* 

.  , 

. 

C««^ 

6-38 


POLYNOMIAL  AND  MULTIVARIABLE  RELATIONSHIPS  AMCP  706-110 


The  elements  c„  are  computed  from  the  following  (note  that  c<}  =  0  for  j  >  t): 

Cn  =  Van 
Cn  —  O11/C11 


Cnl  Onl/Cll 

Cn  =  V<hi  —  cl, 

c«  =  (Ojj  —  C|1  Cji)/Cm 


Cn2  —  (Ctnl  Cnl  Cj,)/Cn 

Cj>  —  V^flyy  C*-,y_i  cJ,y_j  ...  —  cj,, 


C»y  (Oyy  Cyty_,Cyfy_i  Cf"  ty_jC  y  t  y_,  •  *  *  —  Ct,,Cy,i)/Cyy ■ 

As  an  example,  consider 


N  = 

“  4 

6 

8 

10 

6 

25 

20 

27 

8 

20 

36 

30 

10 

27 

30 

36 

Applying  the  formulas  for  c,y,  we  get 

Cn  =  V4  =  2 
Cj,  =  6/2*3 
Ci,  =8/2*4 
e„  -  10/2  =  6 

Cj*  *  \/25  -  (3)a  =  4 
Cm  -  [20  -  4(3)  J/4  =  2 
c„  =  (27  -  6(3) j/4  =  3 
c«  =  V36  -  2*  -  4*  =  4 
c«  =  [30  -  3(2)  -  5(4)  ]/4  *  1 
c«  =  a/36  -  l1  -  32  -  5l  *  1. 

This  gives 


N  = 

"2 

”2  3  4  5' 

3  4 

4  2  3 

4  2  4 

4  1 

_5  3  1  1_ 

1_ 

6-39 


AMCP  706-110 


ANALYSIS  OF  MEASUREMENT  DATA 


The  inverse  of  a  triangular  matrix 


T 


«« 


c„ 


L  c.i 


is  given  by 

fp-i  _ 


c.t 


bu 

6,i 


G..J 


6«i  6»*  .  .  6„ J 

where 


6.1  =  -  (6„«,i)/Ca 

6.1  =  —  (c,i  6n  +  c„  6,i)/cm 


6»i  =  —  (c»i  6u  +  <;»,  6n  +  .  . .  -f  Cr,r-i  6»_i,i)/Crr 


6«  =  —  Cm  6„/c„ 

6«,  =  —  (c«  6„  +  *«  6u)/c44 


6r,  =  —  (Cr»  6m  +  Cnt  6m  +  .  .  .  +  Cr,r_i  6r_i,,)/Crr 


6 ij  —  —  (C.y  bjj  -j-  C<,y+i  6>+i,f  +  . . .  +  C,-.<_i  6<_,,y)/c«. 


Example: 
For  T 


"2 

3 

4 

5 


4 

2 

3 


4 

1 


1J 
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The  elements  of  T~!  are 
bn  -  | 


t.1  -  - 1  •  1 
6”  *  ~  [4  (; 

3 

"  8 
i)  +  2( 

f-fV 
(  8/ 

»IS 

1 

1 

i)  +  s| 

<  s) 

i 

+ 

K-\ 

h-  -  "2  (l) 

1/4  -  - 

1 

‘  8 

*«-  -[»(| 

D+1' 

(-1) 

‘--i 

*■  ■  -1(j) 

t 

6*4  “■  1 . 

Thus,  T~l  -  ± 

8 

-6 

4 

-i 

-5 

-2 

4 

—17 

-10 

—4  16_ 

and  N->  -  (TT')-1  -  (T'-1)  (T->)  give* 


“8  -6  -6  -17“ 

1 

"  8 

1 

414  166  48  -272“ 

4  -2  -10 

16 

-6  4 

“  266 

156  120  82  -160 

4  -4 

-6  -2  4 

48  82  82  -  64 

16. 

_  — 17  -10  -4  16_ 

-272  -160  -  64  256_ 

6-9.3  REMARKS 

By  forming  the  matrix  product 


X'l  (X,Y)  -  [X'X  X'Y~ 
Y* J  L1"*  Y'Y_ 
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and  replacing  Y'X  by  0  (a  null  matrix)  and  Y'Y  by  I  (the  identity  matrix),  we  obtain 


N  = 


X'X  X 
0 


n 


In  this  form,  Y  may  be  a  single  vector  of  observations  Y'  =  (YjY*  . . .  Y„),  or  a  set  of  p  vectors 


Y  = 


Yu 

YS1 


Yb1 


Y.p"' 

Y,p 


Yn  pj. 


Then, 


N~l 2 3 4 5  = 


1  =  r  (X'X)-1  —ff 

o  7 


where  lisp  X  p  and  0  is  p  X  k,  gives  all  the  values  needed  for  the  computations  of  this  Paragraph. 
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